Starting your Machine Learning Project
+Starting your Machine Learning Project
- Identify problem type: classification, generation, regression @@ -312,7 +299,7 @@ Where to find recent results:
- Supervised? @@ -323,7 +310,7 @@ Where to find recent results:
- Shuffle your data @@ -353,7 +340,7 @@ Where to find recent results:
- RELU? ELU? @@ -376,7 +363,7 @@ Where to find recent results:
- Stochastic gradient descent
@@ -401,7 +388,7 @@ Which regularization and hyperparameters? \( L_1 \) or \( L_2 \), soft classifie
- Resampling
+Resampling
When do we resample? @@ -415,18 +402,17 @@ When do we resample?
- + + +Other courses on Data science and Machine Learning at UiO
+Other courses on Data science and Machine Learning at UiO
The link here https://www.mn.uio.no/english/research/about/centre-focus/innovation/data-science/studies/ gives an excellent overview of courses on Machine learning at UiO.
- STK2100 Machine learning and statistical methods for prediction and classification. -
- IN3050 Introduction to Artificial Intelligence and Machine Learning. Introductory course in machine learning and AI with an algorithmic approach. +
- IN3050/IN4050 Introduction to Artificial Intelligence and Machine Learning. Introductory course in machine learning and AI with an algorithmic approach.
- STK-INF3000/4000 Selected Topics in Data Science. The course provides insight into selected contemporary relevant topics within Data Science.
- IN4080 Natural Language Processing. Probabilistic and machine learning techniques applied to natural language processing.
- STK-IN4300 – Statistical learning methods in Data Science. An advanced introduction to statistical and machine learning. For students with a good mathematics and statistics background. -
- INF4490 Biologically Inspired Computing. An introduction to self-adapting methods also called artificial intelligence or machine learning.
- IN-STK5000 Adaptive Methods for Data-Based Decision Making. Methods for adaptive collection and processing of data based on machine learning techniques.
- IN5400/INF5860 – Machine Learning for Image Analysis. An introduction to deep learning with particular emphasis on applications within Image analysis, but useful for other application areas too.
- TEK5040 – Dyp læring for autonome systemer. The course addresses advanced algorithms and architectures for deep learning with neural networks. The course provides an introduction to how deep-learning techniques can be used in the construction of key parts of advanced autonomous systems that exist in physical environments and cyber environments. @@ -435,7 +421,7 @@ The link here Additional courses of interest +
- STK4051 Computational Statistics @@ -445,7 +431,290 @@ The link here Best wishes to you all and thanks so much for your heroic efforts this semester +
- Boosting techniques and complex neural networks +
- Adversarial examples +
- Zero shot learning +
- Transfer learning +
- Model interpretability +
- Convolutional neural networks, which are mostly used in image and video data processing, and have also been applied to sequential data such as text processing; +
- Recurrent neural networks, which can process sequential data of variable length and have been widely used in natural language understanding and speech processing; +
- Encoder-decoder framework, which is mostly used for image or sequence generation, such as machine translation, text summarization, and image captioning. +
Additional courses of interest
Hot Topics Now
+ +-
+
+ +Based on multi-layer nonlinear neural networks, deep learning can +learn directly from raw data, automatically extract and abstract +features from layer to layer, and then achieve the goal of regression, +classification, or ranking. Deep learning has made breakthroughs in +computer vision, speech processing and natural language, and reached +or even surpassed human level. The success of deep learning is mainly +due to the three factors: big data, big model, and big computing. + +
+In the past few decades, many different architectures of deep neural +networks have been proposed, such as + +
-
+
+ + + +Reinforcement Learning
+ ++Reinforcement learning is a sub-area of machine learning. It studies +how agents take actions based on trial and error, so as to maximize +some notion of cumulative reward in a dynamic system or +environment. Due to its generality, the problem has also been studied +in many other disciplines, such as game theory, control theory, +operations research, information theory, multi-agent systems, swarm +intelligence, statistics, and genetic algorithms. + +
+In March 2016, AlphaGo, a computer program that plays the board game +Go, beat Lee Sedol in a five-game match. This was the first time a +computer Go program had beaten a 9-dan (highest rank) professional +without handicaps. AlphaGo is based on deep convolutional neural +networks and reinforcement learning. AlphaGo’s victory was a major +milestone in artificial intelligence and it has also made +reinforcement learning a hot research area in the field of machine +learning. +
+ + + +Transfer learning
+ ++The goal of transfer learning is to transfer the model or knowledge +obtained from a source task to the target task, in order to resolve +the issues of insufficient training data in the target task. The +rationality of doing so lies in that usually the source and target +tasks have inter-correlations, and therefore either the features, +samples, or models in the source task might provide useful information +for us to better solve the target task. Transfer learning is a hot +research topic in recent years, with many problems still waiting to be +solved in this space. +
+ + + +Adversarial learning
+ ++The conventional deep generative model has a potential problem: the +model tends to generate extreme instances to maximize the +probabilistic likelihood, which will hurt its performance. Adversarial +learning utilizes the adversarial behaviors (e.g., generating +adversarial instances or training an adversarial model) to enhance the +robustness of the model and improve the quality of the generated +data. In recent years, one of the most promising unsupervised learning +technologies, generative adversarial networks (GAN), has already been +successfully applied to image, speech, and text. +
+ + + +Dual learning
+ ++Dual learning is a new learning paradigm, the basic idea of which is +to use the primal-dual structure between machine learning tasks to +obtain effective feedback/regularization, and guide and strengthen the +learning process, thus reducing the requirement of large-scale labeled +data for deep learning. The idea of dual learning has been applied to +many problems in machine learning, including machine translation, +image style conversion, question answering and generation, image +classification and generation, text classification and generation, +image-to-text, and text-to-image. +
+ + + +Distributed machine learning
+ ++Distributed computation will speed up machine learning algorithms, +significantly improve their efficiency, and thus enlarge their +application. When distributed meets machine learning, more than just +implementing the machine learning algorithms in parallel is required. +
+ + + +Meta learning
+ ++Meta learning is an emerging research direction in machine +learning. Roughly speaking, meta learning concerns learning how to +learn, and focuses on the understanding and adaptation of the learning +itself, instead of just completing a specific learning task. That is, +a meta learner needs to be able to evaluate its own learning methods +and adjust its own learning methods according to specific learning +tasks. +
+ + + +The Challenges Facing Machine Learning
+ ++While there has been much progress in machine learning, there are also challenges. + +
+For example, the mainstream machine learning technologies are +black-box approaches, making us concerned about their potential +risks. To tackle this challenge, we may want to make machine learning +more explainable and controllable. As another example, the +computational complexity of machine learning algorithms is usually +very high and we may want to invent lightweight algorithms or +implementations. Furthermore, in many domains such as physics, +chemistry, biology, and social sciences, people usually seek elegantly +simple equations (e.g., the Schrödinger equation) to uncover the +underlying laws behind various phenomena. In the field of machine +learning, can we reveal simple laws instead of designing more complex +models for data fitting? Although there are many challenges, we are +still very optimistic about the future of machine learning. As we look +forward to the future, here are what we think the research hotspots in +the next ten years will be. +
+ + + +Explainable machine learning
+ ++Machine learning, especially deep learning, evolves rapidly. The +ability gap between machine and human on many complex cognitive tasks +becomes narrower and narrower. However, we are still in the very early +stage in terms of explaining why those effective models work and how +they work. + +
+What is missing: the gap between correlation and causation Most +machine learning techniques, especially the statistical ones, depend +highly on data correlation to make predictions and analyses. In +contrast, rational humans tend to reply on clear and trustworthy +causality relations obtained via logical reasoning on real and clear +facts. It is one of the core goals of explainable machine learning to +transition from solving problems by data correlation to solving +problems by logical reasoning. +
+ + + +Quantum machine learning
+ ++Quantum machine learning is an emerging interdisciplinary research +area at the intersection of quantum computing and machine learning. + +
+Quantum computers use effects such as quantum coherence and quantum +entanglement to process information, which is fundamentally different +from classical computers. Quantum algorithms have surpassed the best +classical algorithms in several problems (e.g., searching for an +unsorted database, inverting a sparse matrix), which we call quantum +acceleration. + +
+When quantum computing meets machine learning, it can be a mutually +beneficial and reinforcing process, as it allows us to take advantage +of quantum computing to improve the performance of classical machine +learning algorithms. In addition, we can also use the machine learning +algorithms (on classic computers) to analyze and improve quantum +computing systems. +
+ + + +Quantum machine learning algorithms based on linear algebra
+ ++Many quantum machine learning algorithms are based on variants of +quantum algorithms for solving linear equations, which can efficiently +solve N-variable linear equations with complexity of O(log2 N) under +certain conditions. The quantum matrix inversion algorithm can +accelerate many machine learning methods, such as least square linear +regression, least square version of support vector machine, Gaussian +process, and more. The training of these algorithms can be simplified +to solve linear equations. The key bottleneck of this type of quantum +machine learning algorithms is data input—that is, how to initialize +the quantum system with the entire data set. Although efficient +data-input algorithms exist for certain situations, how to efficiently +input data into a quantum system is as yet unknown for most cases. +
+ + + +Quantum reinforcement learning
+ ++In quantum reinforcement learning, a quantum agent interacts with the +classical environment to obtain rewards from the environment, so as to +adjust and improve its behavioral strategies. In some cases, it +achieves quantum acceleration by the quantum processing capabilities +of the agent or the possibility of exploring the environment through +quantum superposition. Such algorithms have been proposed in +superconducting circuits and systems of trapped ions. +
+ + + +Quantum deep learning
+ ++Dedicated quantum information processors, such as quantum annealers +and programmable photonic circuits, are well suited for building deep +quantum networks. The simplest deep quantum network is the Boltzmann +machine. The classical Boltzmann machine consists of bits with tunable +interactions and is trained by adjusting the interaction of these bits +so that the distribution of its expression conforms to the statistics +of the data. To quantize the Boltzmann machine, the neural network can +simply be represented as a set of interacting quantum spins that +correspond to an adjustable Ising model. Then, by initializing the +input neurons in the Boltzmann machine to a fixed state and allowing +the system to heat up, we can read out the output qubits to get the +result. +
+ + + +Social machine learning
+ ++Machine learning aims to imitate how humans +learn. While we have developed successful machine learning algorithms, +until now we have ignored one important fact: humans are social. Each +of us is one part of the total society and it is difficult for us to +live, learn, and improve ourselves, alone and isolated. Therefore, we +should design machines with social properties. Can we let machines +evolve by imitating human society so as to achieve more effective, +intelligent, interpretable “social machine learning”? + +
+And much more. +
+ + + +The last words?
+ ++Early computer scientist Alan Kay said, The best way to predict the +future is to create it. Therefore, all machine learning +practitioners, whether scholars or engineers, professors or students, +need to work together to advance these important research +topics. Together, we will not just predict the future, but create it. +
+ Best wishes to you all and thanks so much for your heroic efforts this semester
diff --git a/doc/pub/summary/html/summary-solarized.html b/doc/pub/summary/html/summary-solarized.html index 8e005268d..dc99907f6 100644 --- a/doc/pub/summary/html/summary-solarized.html +++ b/doc/pub/summary/html/summary-solarized.html @@ -52,26 +52,43 @@ div { text-align: justify; text-justify: inter-word; } '___sec5'), ('Perspective on Machine Learning', 2, None, '___sec6'), ('Machine Learning Research', 2, None, '___sec7'), - ('Hot Topics Now', 2, None, '___sec8'), - ('Starting your Machine Learning Project', 2, None, '___sec9'), - ('Choose a Model and Algorithm', 2, None, '___sec10'), - ('Preparing Your Data', 2, None, '___sec11'), + ('Starting your Machine Learning Project', 2, None, '___sec8'), + ('Choose a Model and Algorithm', 2, None, '___sec9'), + ('Preparing Your Data', 2, None, '___sec10'), ('Which Activation and Weights to Choose in Neural Networks', 2, None, - '___sec12'), - ('Optimization Methods and Hyperparameters', 2, None, '___sec13'), - ('Resampling', 2, None, '___sec14'), + '___sec11'), + ('Optimization Methods and Hyperparameters', 2, None, '___sec12'), + ('Resampling', 2, None, '___sec13'), ('Other courses on Data science and Machine Learning at UiO', 2, None, - '___sec15'), - ('Additional courses of interest', 2, None, '___sec16'), + '___sec14'), + ('Additional courses of interest', 2, None, '___sec15'), + ('Hot Topics Now', 2, None, '___sec16'), + ('Reinforcement Learning', 2, None, '___sec17'), + ('Transfer learning', 2, None, '___sec18'), + ('Adversarial learning', 2, None, '___sec19'), + ('Dual learning', 2, None, '___sec20'), + ('Distributed machine learning', 2, None, '___sec21'), + ('Meta learning', 2, None, '___sec22'), + ('The Challenges Facing Machine Learning', 2, None, '___sec23'), + ('Explainable machine learning', 2, None, '___sec24'), + ('Quantum machine learning', 2, None, '___sec25'), + ('Quantum machine learning algorithms based on linear algebra', + 2, + None, + '___sec26'), + ('Quantum reinforcement learning', 2, None, '___sec27'), + ('Quantum deep learning', 2, None, '___sec28'), + ('Social machine learning', 2, None, '___sec29'), + ('The last words?', 2, None, '___sec30'), ('Best wishes to you all and thanks so much for your heroic ' 'efforts this semester', 2, None, - '___sec17')]} + '___sec31')]} end of tocinfo --> @@ -239,19 +256,7 @@ Where to find recent results:
-Hot Topics Now
- --
-
- Boosting techniques and complex neural networks -
- Adversarial examples -
- Zero shot learning -
- Transfer learning -
- Model interpretability -
- -Starting your Machine Learning Project
+Starting your Machine Learning Project
- Identify problem type: classification, generation, regression @@ -263,7 +268,7 @@ Where to find recent results:
- Supervised? @@ -273,7 +278,7 @@ Where to find recent results:
- Shuffle your data @@ -301,7 +306,7 @@ Where to find recent results:
- RELU? ELU? @@ -322,7 +327,7 @@ Where to find recent results:
- Stochastic gradient descent
@@ -345,7 +350,7 @@ Which regularization and hyperparameters? \( L_1 \) or \( L_2 \), soft classifie
-Resampling
+Resampling
When do we resample? @@ -358,18 +363,17 @@ When do we resample?
-Other courses on Data science and Machine Learning at UiO
+Other courses on Data science and Machine Learning at UiO
The link here https://www.mn.uio.no/english/research/about/centre-focus/innovation/data-science/studies/ gives an excellent overview of courses on Machine learning at UiO.
- STK2100 Machine learning and statistical methods for prediction and classification. -
- IN3050 Introduction to Artificial Intelligence and Machine Learning. Introductory course in machine learning and AI with an algorithmic approach. +
- IN3050/IN4050 Introduction to Artificial Intelligence and Machine Learning. Introductory course in machine learning and AI with an algorithmic approach.
- STK-INF3000/4000 Selected Topics in Data Science. The course provides insight into selected contemporary relevant topics within Data Science.
- IN4080 Natural Language Processing. Probabilistic and machine learning techniques applied to natural language processing.
- STK-IN4300 – Statistical learning methods in Data Science. An advanced introduction to statistical and machine learning. For students with a good mathematics and statistics background. -
- INF4490 Biologically Inspired Computing. An introduction to self-adapting methods also called artificial intelligence or machine learning.
- IN-STK5000 Adaptive Methods for Data-Based Decision Making. Methods for adaptive collection and processing of data based on machine learning techniques.
- IN5400/INF5860 – Machine Learning for Image Analysis. An introduction to deep learning with particular emphasis on applications within Image analysis, but useful for other application areas too.
- TEK5040 – Dyp læring for autonome systemer. The course addresses advanced algorithms and architectures for deep learning with neural networks. The course provides an introduction to how deep-learning techniques can be used in the construction of key parts of advanced autonomous systems that exist in physical environments and cyber environments. @@ -377,7 +381,7 @@ The link here Additional courses of interest +
- STK4051 Computational Statistics @@ -386,13 +390,292 @@ The link here Best wishes to you all and thanks so much for your heroic efforts this semester +
- Boosting techniques and complex neural networks +
- Adversarial examples +
- Zero shot learning +
- Transfer learning +
- Model interpretability +
- Convolutional neural networks, which are mostly used in image and video data processing, and have also been applied to sequential data such as text processing; +
- Recurrent neural networks, which can process sequential data of variable length and have been widely used in natural language understanding and speech processing; +
- Encoder-decoder framework, which is mostly used for image or sequence generation, such as machine translation, text summarization, and image captioning. +
- Boosting techniques and complex neural networks -
- Adversarial examples -
- Zero shot learning -
- Transfer learning -
- Model interpretability -
- Identify problem type: classification, generation, regression @@ -268,7 +273,7 @@ Where to find recent results:
- Supervised? @@ -278,7 +283,7 @@ Where to find recent results:
- Shuffle your data @@ -306,7 +311,7 @@ Where to find recent results:
- RELU? ELU? @@ -327,7 +332,7 @@ Where to find recent results:
- Stochastic gradient descent
@@ -350,7 +355,7 @@ Which regularization and hyperparameters? \( L_1 \) or \( L_2 \), soft classifie
-Resampling
+Resampling
When do we resample? @@ -363,18 +368,17 @@ When do we resample?
-Other courses on Data science and Machine Learning at UiO
+Other courses on Data science and Machine Learning at UiO
The link here https://www.mn.uio.no/english/research/about/centre-focus/innovation/data-science/studies/ gives an excellent overview of courses on Machine learning at UiO.
- STK2100 Machine learning and statistical methods for prediction and classification. -
- IN3050 Introduction to Artificial Intelligence and Machine Learning. Introductory course in machine learning and AI with an algorithmic approach. +
- IN3050/IN4050 Introduction to Artificial Intelligence and Machine Learning. Introductory course in machine learning and AI with an algorithmic approach.
- STK-INF3000/4000 Selected Topics in Data Science. The course provides insight into selected contemporary relevant topics within Data Science.
- IN4080 Natural Language Processing. Probabilistic and machine learning techniques applied to natural language processing.
- STK-IN4300 – Statistical learning methods in Data Science. An advanced introduction to statistical and machine learning. For students with a good mathematics and statistics background. -
- INF4490 Biologically Inspired Computing. An introduction to self-adapting methods also called artificial intelligence or machine learning.
- IN-STK5000 Adaptive Methods for Data-Based Decision Making. Methods for adaptive collection and processing of data based on machine learning techniques.
- IN5400/INF5860 – Machine Learning for Image Analysis. An introduction to deep learning with particular emphasis on applications within Image analysis, but useful for other application areas too.
- TEK5040 – Dyp læring for autonome systemer. The course addresses advanced algorithms and architectures for deep learning with neural networks. The course provides an introduction to how deep-learning techniques can be used in the construction of key parts of advanced autonomous systems that exist in physical environments and cyber environments. @@ -382,7 +386,7 @@ The link here Additional courses of interest +
- STK4051 Computational Statistics @@ -391,13 +395,292 @@ The link here Best wishes to you all and thanks so much for your heroic efforts this semester +
- Boosting techniques and complex neural networks +
- Adversarial examples +
- Zero shot learning +
- Transfer learning +
- Model interpretability +
- Convolutional neural networks, which are mostly used in image and video data processing, and have also been applied to sequential data such as text processing; +
- Recurrent neural networks, which can process sequential data of variable length and have been widely used in natural language understanding and speech processing; +
- Encoder-decoder framework, which is mostly used for image or sequence generation, such as machine translation, text summarization, and image captioning. +
Additional courses of interest
Hot Topics Now
+ +-
+
+In the past few decades, many different architectures of deep neural +networks have been proposed, such as + +
-
+
+ +Reinforcement Learning
+ ++Reinforcement learning is a sub-area of machine learning. It studies +how agents take actions based on trial and error, so as to maximize +some notion of cumulative reward in a dynamic system or +environment. Due to its generality, the problem has also been studied +in many other disciplines, such as game theory, control theory, +operations research, information theory, multi-agent systems, swarm +intelligence, statistics, and genetic algorithms. + +
+In March 2016, AlphaGo, a computer program that plays the board game +Go, beat Lee Sedol in a five-game match. This was the first time a +computer Go program had beaten a 9-dan (highest rank) professional +without handicaps. AlphaGo is based on deep convolutional neural +networks and reinforcement learning. AlphaGo’s victory was a major +milestone in artificial intelligence and it has also made +reinforcement learning a hot research area in the field of machine +learning. + +
+
+ +Transfer learning
+ ++The goal of transfer learning is to transfer the model or knowledge +obtained from a source task to the target task, in order to resolve +the issues of insufficient training data in the target task. The +rationality of doing so lies in that usually the source and target +tasks have inter-correlations, and therefore either the features, +samples, or models in the source task might provide useful information +for us to better solve the target task. Transfer learning is a hot +research topic in recent years, with many problems still waiting to be +solved in this space. + +
+
+ +Adversarial learning
+ ++The conventional deep generative model has a potential problem: the +model tends to generate extreme instances to maximize the +probabilistic likelihood, which will hurt its performance. Adversarial +learning utilizes the adversarial behaviors (e.g., generating +adversarial instances or training an adversarial model) to enhance the +robustness of the model and improve the quality of the generated +data. In recent years, one of the most promising unsupervised learning +technologies, generative adversarial networks (GAN), has already been +successfully applied to image, speech, and text. + +
+
+ +Dual learning
+ ++Dual learning is a new learning paradigm, the basic idea of which is +to use the primal-dual structure between machine learning tasks to +obtain effective feedback/regularization, and guide and strengthen the +learning process, thus reducing the requirement of large-scale labeled +data for deep learning. The idea of dual learning has been applied to +many problems in machine learning, including machine translation, +image style conversion, question answering and generation, image +classification and generation, text classification and generation, +image-to-text, and text-to-image. + +
+
+ +Distributed machine learning
+ ++Distributed computation will speed up machine learning algorithms, +significantly improve their efficiency, and thus enlarge their +application. When distributed meets machine learning, more than just +implementing the machine learning algorithms in parallel is required. + +
+
+ +Meta learning
+ ++Meta learning is an emerging research direction in machine +learning. Roughly speaking, meta learning concerns learning how to +learn, and focuses on the understanding and adaptation of the learning +itself, instead of just completing a specific learning task. That is, +a meta learner needs to be able to evaluate its own learning methods +and adjust its own learning methods according to specific learning +tasks. + +
+
+ +The Challenges Facing Machine Learning
+ ++While there has been much progress in machine learning, there are also challenges. + +
+For example, the mainstream machine learning technologies are +black-box approaches, making us concerned about their potential +risks. To tackle this challenge, we may want to make machine learning +more explainable and controllable. As another example, the +computational complexity of machine learning algorithms is usually +very high and we may want to invent lightweight algorithms or +implementations. Furthermore, in many domains such as physics, +chemistry, biology, and social sciences, people usually seek elegantly +simple equations (e.g., the Schrödinger equation) to uncover the +underlying laws behind various phenomena. In the field of machine +learning, can we reveal simple laws instead of designing more complex +models for data fitting? Although there are many challenges, we are +still very optimistic about the future of machine learning. As we look +forward to the future, here are what we think the research hotspots in +the next ten years will be. + +
+
+ +Explainable machine learning
+ ++Machine learning, especially deep learning, evolves rapidly. The +ability gap between machine and human on many complex cognitive tasks +becomes narrower and narrower. However, we are still in the very early +stage in terms of explaining why those effective models work and how +they work. + +
+What is missing: the gap between correlation and causation Most +machine learning techniques, especially the statistical ones, depend +highly on data correlation to make predictions and analyses. In +contrast, rational humans tend to reply on clear and trustworthy +causality relations obtained via logical reasoning on real and clear +facts. It is one of the core goals of explainable machine learning to +transition from solving problems by data correlation to solving +problems by logical reasoning. + +
+
+ +Quantum machine learning
+ ++Quantum machine learning is an emerging interdisciplinary research +area at the intersection of quantum computing and machine learning. + +
+Quantum computers use effects such as quantum coherence and quantum +entanglement to process information, which is fundamentally different +from classical computers. Quantum algorithms have surpassed the best +classical algorithms in several problems (e.g., searching for an +unsorted database, inverting a sparse matrix), which we call quantum +acceleration. + +
+When quantum computing meets machine learning, it can be a mutually +beneficial and reinforcing process, as it allows us to take advantage +of quantum computing to improve the performance of classical machine +learning algorithms. In addition, we can also use the machine learning +algorithms (on classic computers) to analyze and improve quantum +computing systems. + +
+
+ +Quantum machine learning algorithms based on linear algebra
+ ++Many quantum machine learning algorithms are based on variants of +quantum algorithms for solving linear equations, which can efficiently +solve N-variable linear equations with complexity of O(log2 N) under +certain conditions. The quantum matrix inversion algorithm can +accelerate many machine learning methods, such as least square linear +regression, least square version of support vector machine, Gaussian +process, and more. The training of these algorithms can be simplified +to solve linear equations. The key bottleneck of this type of quantum +machine learning algorithms is data input—that is, how to initialize +the quantum system with the entire data set. Although efficient +data-input algorithms exist for certain situations, how to efficiently +input data into a quantum system is as yet unknown for most cases. + +
+
+ +Quantum reinforcement learning
+ ++In quantum reinforcement learning, a quantum agent interacts with the +classical environment to obtain rewards from the environment, so as to +adjust and improve its behavioral strategies. In some cases, it +achieves quantum acceleration by the quantum processing capabilities +of the agent or the possibility of exploring the environment through +quantum superposition. Such algorithms have been proposed in +superconducting circuits and systems of trapped ions. + +
+
+ +Quantum deep learning
+ ++Dedicated quantum information processors, such as quantum annealers +and programmable photonic circuits, are well suited for building deep +quantum networks. The simplest deep quantum network is the Boltzmann +machine. The classical Boltzmann machine consists of bits with tunable +interactions and is trained by adjusting the interaction of these bits +so that the distribution of its expression conforms to the statistics +of the data. To quantize the Boltzmann machine, the neural network can +simply be represented as a set of interacting quantum spins that +correspond to an adjustable Ising model. Then, by initializing the +input neurons in the Boltzmann machine to a fixed state and allowing +the system to heat up, we can read out the output qubits to get the +result. + +
+
+ +Social machine learning
+ ++Machine learning aims to imitate how humans +learn. While we have developed successful machine learning algorithms, +until now we have ignored one important fact: humans are social. Each +of us is one part of the total society and it is difficult for us to +live, learn, and improve ourselves, alone and isolated. Therefore, we +should design machines with social properties. Can we let machines +evolve by imitating human society so as to achieve more effective, +intelligent, interpretable “social machine learning”? + +
+And much more. + +
+
+ +The last words?
+ ++Early computer scientist Alan Kay said, The best way to predict the +future is to create it. Therefore, all machine learning +practitioners, whether scholars or engineers, professors or students, +need to work together to advance these important research +topics. Together, we will not just predict the future, but create it. + +
+
+ +Best wishes to you all and thanks so much for your heroic efforts this semester
-- diff --git a/doc/pub/summary/ipynb/ipynb-summary-src.tar.gz b/doc/pub/summary/ipynb/ipynb-summary-src.tar.gz index e90ff5991..dbc79daf1 100644 Binary files a/doc/pub/summary/ipynb/ipynb-summary-src.tar.gz and b/doc/pub/summary/ipynb/ipynb-summary-src.tar.gz differ diff --git a/doc/pub/summary/ipynb/summary.ipynb b/doc/pub/summary/ipynb/summary.ipynb index 6f613f755..3cea2fa8d 100644 --- a/doc/pub/summary/ipynb/summary.ipynb +++ b/doc/pub/summary/ipynb/summary.ipynb @@ -146,18 +146,6 @@ "\n", "5. [Journal of Machine Learning Research](http://www.jmlr.org/papers/v19/) \n", "\n", - "## Hot Topics Now\n", - "\n", - "1. Boosting techniques and complex neural networks\n", - "\n", - "2. [Adversarial examples](https://medium.com/@ml.at.berkeley/trickingneural-networks-create-your-own-adversarial-examples-a61eb7620fd8)\n", - "\n", - "3. [Zero shot learning](https://arxiv.org/pdf/1707.00600)\n", - "\n", - "4. Transfer learning\n", - "\n", - "5. [Model interpretability](https://christophm.github.io/interpretable-mlbook/interpretability.html)\n", - "\n", "## Starting your Machine Learning Project\n", "\n", "1. Identify problem type: classification, generation, regression\n", @@ -249,7 +237,7 @@ "\n", "1. [STK2100 Machine learning and statistical methods for prediction and classification](http://www.uio.no/studier/emner/matnat/math/STK2100/index-eng.html). \n", "\n", - "2. [IN3050 Introduction to Artificial Intelligence and Machine Learning](https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html). Introductory course in machine learning and AI with an algorithmic approach. \n", + "2. [IN3050/IN4050 Introduction to Artificial Intelligence and Machine Learning](https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html). Introductory course in machine learning and AI with an algorithmic approach. \n", "\n", "3. [STK-INF3000/4000 Selected Topics in Data Science](http://www.uio.no/studier/emner/matnat/math/STK-INF3000/index-eng.html). The course provides insight into selected contemporary relevant topics within Data Science. \n", "\n", @@ -257,13 +245,11 @@ "\n", "5. [STK-IN4300 – Statistical learning methods in Data Science](https://www.uio.no/studier/emner/matnat/math/STK-IN4300/index-eng.html). An advanced introduction to statistical and machine learning. For students with a good mathematics and statistics background.\n", "\n", - "6. [INF4490 Biologically Inspired Computing](http://www.uio.no/studier/emner/matnat/ifi/INF4490/). An introduction to self-adapting methods also called artificial intelligence or machine learning. \n", + "6. [IN-STK5000 Adaptive Methods for Data-Based Decision Making](https://www.uio.no/studier/emner/matnat/ifi/IN-STK5000/index-eng.html). Methods for adaptive collection and processing of data based on machine learning techniques. \n", "\n", - "7. [IN-STK5000 Adaptive Methods for Data-Based Decision Making](https://www.uio.no/studier/emner/matnat/ifi/IN-STK5000/index-eng.html). Methods for adaptive collection and processing of data based on machine learning techniques. \n", + "7. [IN5400/INF5860 – Machine Learning for Image Analysis](https://www.uio.no/studier/emner/matnat/ifi/IN5400/). An introduction to deep learning with particular emphasis on applications within Image analysis, but useful for other application areas too.\n", "\n", - "8. [IN5400/INF5860 – Machine Learning for Image Analysis](https://www.uio.no/studier/emner/matnat/ifi/IN5400/). An introduction to deep learning with particular emphasis on applications within Image analysis, but useful for other application areas too.\n", - "\n", - "9. [TEK5040 – Dyp læring for autonome systemer](https://www.uio.no/studier/emner/matnat/its/TEK5040/). The course addresses advanced algorithms and architectures for deep learning with neural networks. The course provides an introduction to how deep-learning techniques can be used in the construction of key parts of advanced autonomous systems that exist in physical environments and cyber environments.\n", + "8. [TEK5040 – Dyp læring for autonome systemer](https://www.uio.no/studier/emner/matnat/its/TEK5040/). The course addresses advanced algorithms and architectures for deep learning with neural networks. The course provides an introduction to how deep-learning techniques can be used in the construction of key parts of advanced autonomous systems that exist in physical environments and cyber environments.\n", "\n", "## Additional courses of interest\n", "\n", @@ -271,6 +257,231 @@ "\n", "2. [STK4021 Applied Bayesian Analysis and Numerical Methods](https://www.uio.no/studier/emner/matnat/math/STK4021/index-eng.html)\n", "\n", + "## Hot Topics Now\n", + "\n", + "1. Boosting techniques and complex neural networks\n", + "\n", + "2. [Adversarial examples](https://medium.com/@ml.at.berkeley/trickingneural-networks-create-your-own-adversarial-examples-a61eb7620fd8)\n", + "\n", + "3. [Zero shot learning](https://arxiv.org/pdf/1707.00600)\n", + "\n", + "4. Transfer learning\n", + "\n", + "5. [Model interpretability](https://christophm.github.io/interpretable-mlbook/interpretability.html)\n", + "\n", + "Based on multi-layer nonlinear neural networks, deep learning can\n", + "learn directly from raw data, automatically extract and abstract\n", + "features from layer to layer, and then achieve the goal of regression,\n", + "classification, or ranking. Deep learning has made breakthroughs in\n", + "computer vision, speech processing and natural language, and reached\n", + "or even surpassed human level. The success of deep learning is mainly\n", + "due to the three factors: big data, big model, and big computing.\n", + "\n", + "In the past few decades, many different architectures of deep neural\n", + "networks have been proposed, such as\n", + "1. Convolutional neural networks, which are mostly used in image and video data processing, and have also been applied to sequential data such as text processing;\n", + "\n", + "2. Recurrent neural networks, which can process sequential data of variable length and have been widely used in natural language understanding and speech processing;\n", + "\n", + "3. Encoder-decoder framework, which is mostly used for image or sequence generation, such as machine translation, text summarization, and image captioning.\n", + "\n", + "## Reinforcement Learning\n", + "\n", + "Reinforcement learning is a sub-area of machine learning. It studies\n", + "how agents take actions based on trial and error, so as to maximize\n", + "some notion of cumulative reward in a dynamic system or\n", + "environment. Due to its generality, the problem has also been studied\n", + "in many other disciplines, such as game theory, control theory,\n", + "operations research, information theory, multi-agent systems, swarm\n", + "intelligence, statistics, and genetic algorithms.\n", + "\n", + "In March 2016, AlphaGo, a computer program that plays the board game\n", + "Go, beat Lee Sedol in a five-game match. This was the first time a\n", + "computer Go program had beaten a 9-dan (highest rank) professional\n", + "without handicaps. AlphaGo is based on deep convolutional neural\n", + "networks and reinforcement learning. AlphaGo’s victory was a major\n", + "milestone in artificial intelligence and it has also made\n", + "reinforcement learning a hot research area in the field of machine\n", + "learning.\n", + "\n", + "## Transfer learning\n", + "\n", + "The goal of transfer learning is to transfer the model or knowledge\n", + "obtained from a source task to the target task, in order to resolve\n", + "the issues of insufficient training data in the target task. The\n", + "rationality of doing so lies in that usually the source and target\n", + "tasks have inter-correlations, and therefore either the features,\n", + "samples, or models in the source task might provide useful information\n", + "for us to better solve the target task. Transfer learning is a hot\n", + "research topic in recent years, with many problems still waiting to be\n", + "solved in this space.\n", + "\n", + "\n", + "## Adversarial learning\n", + "\n", + "The conventional deep generative model has a potential problem: the\n", + "model tends to generate extreme instances to maximize the\n", + "probabilistic likelihood, which will hurt its performance. Adversarial\n", + "learning utilizes the adversarial behaviors (e.g., generating\n", + "adversarial instances or training an adversarial model) to enhance the\n", + "robustness of the model and improve the quality of the generated\n", + "data. In recent years, one of the most promising unsupervised learning\n", + "technologies, generative adversarial networks (GAN), has already been\n", + "successfully applied to image, speech, and text.\n", + "\n", + "## Dual learning\n", + "\n", + "Dual learning is a new learning paradigm, the basic idea of which is\n", + "to use the primal-dual structure between machine learning tasks to\n", + "obtain effective feedback/regularization, and guide and strengthen the\n", + "learning process, thus reducing the requirement of large-scale labeled\n", + "data for deep learning. The idea of dual learning has been applied to\n", + "many problems in machine learning, including machine translation,\n", + "image style conversion, question answering and generation, image\n", + "classification and generation, text classification and generation,\n", + "image-to-text, and text-to-image.\n", + "\n", + "## Distributed machine learning\n", + "\n", + "Distributed computation will speed up machine learning algorithms,\n", + "significantly improve their efficiency, and thus enlarge their\n", + "application. When distributed meets machine learning, more than just\n", + "implementing the machine learning algorithms in parallel is required.\n", + "\n", + "\n", + "## Meta learning\n", + "\n", + "Meta learning is an emerging research direction in machine\n", + "learning. Roughly speaking, meta learning concerns learning how to\n", + "learn, and focuses on the understanding and adaptation of the learning\n", + "itself, instead of just completing a specific learning task. That is,\n", + "a meta learner needs to be able to evaluate its own learning methods\n", + "and adjust its own learning methods according to specific learning\n", + "tasks.\n", + "\n", + "## The Challenges Facing Machine Learning\n", + "\n", + "While there has been much progress in machine learning, there are also challenges.\n", + "\n", + "For example, the mainstream machine learning technologies are\n", + "black-box approaches, making us concerned about their potential\n", + "risks. To tackle this challenge, we may want to make machine learning\n", + "more explainable and controllable. As another example, the\n", + "computational complexity of machine learning algorithms is usually\n", + "very high and we may want to invent lightweight algorithms or\n", + "implementations. Furthermore, in many domains such as physics,\n", + "chemistry, biology, and social sciences, people usually seek elegantly\n", + "simple equations (e.g., the Schrödinger equation) to uncover the\n", + "underlying laws behind various phenomena. In the field of machine\n", + "learning, can we reveal simple laws instead of designing more complex\n", + "models for data fitting? Although there are many challenges, we are\n", + "still very optimistic about the future of machine learning. As we look\n", + "forward to the future, here are what we think the research hotspots in\n", + "the next ten years will be.\n", + "\n", + "\n", + "## Explainable machine learning\n", + "\n", + "Machine learning, especially deep learning, evolves rapidly. The\n", + "ability gap between machine and human on many complex cognitive tasks\n", + "becomes narrower and narrower. However, we are still in the very early\n", + "stage in terms of explaining why those effective models work and how\n", + "they work.\n", + "\n", + "What is missing: the gap between correlation and causation Most\n", + "machine learning techniques, especially the statistical ones, depend\n", + "highly on data correlation to make predictions and analyses. In\n", + "contrast, rational humans tend to reply on clear and trustworthy\n", + "causality relations obtained via logical reasoning on real and clear\n", + "facts. It is one of the core goals of explainable machine learning to\n", + "transition from solving problems by data correlation to solving\n", + "problems by logical reasoning.\n", + "\n", + "## Quantum machine learning\n", + "\n", + "Quantum machine learning is an emerging interdisciplinary research\n", + "area at the intersection of quantum computing and machine learning.\n", + "\n", + "Quantum computers use effects such as quantum coherence and quantum\n", + "entanglement to process information, which is fundamentally different\n", + "from classical computers. Quantum algorithms have surpassed the best\n", + "classical algorithms in several problems (e.g., searching for an\n", + "unsorted database, inverting a sparse matrix), which we call quantum\n", + "acceleration.\n", + "\n", + "When quantum computing meets machine learning, it can be a mutually\n", + "beneficial and reinforcing process, as it allows us to take advantage\n", + "of quantum computing to improve the performance of classical machine\n", + "learning algorithms. In addition, we can also use the machine learning\n", + "algorithms (on classic computers) to analyze and improve quantum\n", + "computing systems.\n", + "\n", + "\n", + "## Quantum machine learning algorithms based on linear algebra\n", + "\n", + "Many quantum machine learning algorithms are based on variants of\n", + "quantum algorithms for solving linear equations, which can efficiently\n", + "solve N-variable linear equations with complexity of O(log2 N) under\n", + "certain conditions. The quantum matrix inversion algorithm can\n", + "accelerate many machine learning methods, such as least square linear\n", + "regression, least square version of support vector machine, Gaussian\n", + "process, and more. The training of these algorithms can be simplified\n", + "to solve linear equations. The key bottleneck of this type of quantum\n", + "machine learning algorithms is data input—that is, how to initialize\n", + "the quantum system with the entire data set. Although efficient\n", + "data-input algorithms exist for certain situations, how to efficiently\n", + "input data into a quantum system is as yet unknown for most cases.\n", + "\n", + "## Quantum reinforcement learning\n", + "\n", + "In quantum reinforcement learning, a quantum agent interacts with the\n", + "classical environment to obtain rewards from the environment, so as to\n", + "adjust and improve its behavioral strategies. In some cases, it\n", + "achieves quantum acceleration by the quantum processing capabilities\n", + "of the agent or the possibility of exploring the environment through\n", + "quantum superposition. Such algorithms have been proposed in\n", + "superconducting circuits and systems of trapped ions.\n", + "\n", + "## Quantum deep learning\n", + "\n", + "Dedicated quantum information processors, such as quantum annealers\n", + "and programmable photonic circuits, are well suited for building deep\n", + "quantum networks. The simplest deep quantum network is the Boltzmann\n", + "machine. The classical Boltzmann machine consists of bits with tunable\n", + "interactions and is trained by adjusting the interaction of these bits\n", + "so that the distribution of its expression conforms to the statistics\n", + "of the data. To quantize the Boltzmann machine, the neural network can\n", + "simply be represented as a set of interacting quantum spins that\n", + "correspond to an adjustable Ising model. Then, by initializing the\n", + "input neurons in the Boltzmann machine to a fixed state and allowing\n", + "the system to heat up, we can read out the output qubits to get the\n", + "result.\n", + "\n", + "\n", + "## Social machine learning\n", + "\n", + "Machine learning aims to imitate how humans\n", + "learn. While we have developed successful machine learning algorithms,\n", + "until now we have ignored one important fact: humans are social. Each\n", + "of us is one part of the total society and it is difficult for us to\n", + "live, learn, and improve ourselves, alone and isolated. Therefore, we\n", + "should design machines with social properties. Can we let machines\n", + "evolve by imitating human society so as to achieve more effective,\n", + "intelligent, interpretable “social machine learning”?\n", + "\n", + "And much more.\n", + "\n", + "## The last words?\n", + "\n", + "Early computer scientist Alan Kay said, **The best way to predict the\n", + "future is to create it**. Therefore, all machine learning\n", + "practitioners, whether scholars or engineers, professors or students,\n", + "need to work together to advance these important research\n", + "topics. Together, we will not just predict the future, but create it.\n", + "\n", + "\n", + "\n", + "\n", "## Best wishes to you all and thanks so much for your heroic efforts this semester\n", "\n", "\n", diff --git a/doc/pub/summary/pdf/summary-minted.pdf b/doc/pub/summary/pdf/summary-minted.pdf index 54bff9953..50f6829b4 100644 Binary files a/doc/pub/summary/pdf/summary-minted.pdf and b/doc/pub/summary/pdf/summary-minted.pdf differ diff --git a/doc/src/Summary/summary.do.txt b/doc/src/Summary/summary.do.txt index 3f2dde3cd..c70b0f5ee 100644 --- a/doc/src/Summary/summary.do.txt +++ b/doc/src/Summary/summary.do.txt @@ -97,14 +97,6 @@ o _ICLR_: "International Conference on Learning Representations":"https://openre o _ICML_: International Conference on Machine Learning o "Journal of Machine Learning Research":"http://www.jmlr.org/papers/v19/" -!split -===== Hot Topics Now ===== - -o Boosting techniques and complex neural networks -o "Adversarial examples":"https://medium.com/@ml.at.berkeley/trickingneural-networks-create-your-own-adversarial-examples-a61eb7620fd8" -o "Zero shot learning":"https://arxiv.org/pdf/1707.00600" -o Transfer learning -o "Model interpretability":"https://christophm.github.io/interpretable-mlbook/interpretability.html" !split @@ -178,11 +170,10 @@ o Jackknife and many other The link here URL:"https://www.mn.uio.no/english/research/about/centre-focus/innovation/data-science/studies/" gives an excellent overview of courses on Machine learning at UiO. o "STK2100 Machine learning and statistical methods for prediction and classification":"http://www.uio.no/studier/emner/matnat/math/STK2100/index-eng.html". -o "IN3050 Introduction to Artificial Intelligence and Machine Learning":"https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html". Introductory course in machine learning and AI with an algorithmic approach. +o "IN3050/IN4050 Introduction to Artificial Intelligence and Machine Learning":"https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html". Introductory course in machine learning and AI with an algorithmic approach. o "STK-INF3000/4000 Selected Topics in Data Science":"http://www.uio.no/studier/emner/matnat/math/STK-INF3000/index-eng.html". The course provides insight into selected contemporary relevant topics within Data Science. o "IN4080 Natural Language Processing":"https://www.uio.no/studier/emner/matnat/ifi/IN4080/index.html". Probabilistic and machine learning techniques applied to natural language processing. o "STK-IN4300 – Statistical learning methods in Data Science":"https://www.uio.no/studier/emner/matnat/math/STK-IN4300/index-eng.html". An advanced introduction to statistical and machine learning. For students with a good mathematics and statistics background. -o "INF4490 Biologically Inspired Computing":"http://www.uio.no/studier/emner/matnat/ifi/INF4490/". An introduction to self-adapting methods also called artificial intelligence or machine learning. o "IN-STK5000 Adaptive Methods for Data-Based Decision Making":"https://www.uio.no/studier/emner/matnat/ifi/IN-STK5000/index-eng.html". Methods for adaptive collection and processing of data based on machine learning techniques. o "IN5400/INF5860 – Machine Learning for Image Analysis":"https://www.uio.no/studier/emner/matnat/ifi/IN5400/". An introduction to deep learning with particular emphasis on applications within Image analysis, but useful for other application areas too. o "TEK5040 – Dyp læring for autonome systemer":"https://www.uio.no/studier/emner/matnat/its/TEK5040/". The course addresses advanced algorithms and architectures for deep learning with neural networks. The course provides an introduction to how deep-learning techniques can be used in the construction of key parts of advanced autonomous systems that exist in physical environments and cyber environments. @@ -193,12 +184,242 @@ o "TEK5040 – Dyp læring for autonome systemer":"https://www.uio.no/studier/em o "STK4051 Computational Statistics":"https://www.uio.no/studier/emner/matnat/math/STK4051/index-eng.html" o "STK4021 Applied Bayesian Analysis and Numerical Methods":"https://www.uio.no/studier/emner/matnat/math/STK4021/index-eng.html" +!split +===== Hot Topics Now ===== + +o Boosting techniques and complex neural networks +o "Adversarial examples":"https://medium.com/@ml.at.berkeley/trickingneural-networks-create-your-own-adversarial-examples-a61eb7620fd8" +o "Zero shot learning":"https://arxiv.org/pdf/1707.00600" +o Transfer learning +o "Model interpretability":"https://christophm.github.io/interpretable-mlbook/interpretability.html" + + +Based on multi-layer nonlinear neural networks, deep learning can +learn directly from raw data, automatically extract and abstract +features from layer to layer, and then achieve the goal of regression, +classification, or ranking. Deep learning has made breakthroughs in +computer vision, speech processing and natural language, and reached +or even surpassed human level. The success of deep learning is mainly +due to the three factors: big data, big model, and big computing. + +In the past few decades, many different architectures of deep neural +networks have been proposed, such as +o Convolutional neural networks, which are mostly used in image and video data processing, and have also been applied to sequential data such as text processing; +o Recurrent neural networks, which can process sequential data of variable length and have been widely used in natural language understanding and speech processing; +o Encoder-decoder framework, which is mostly used for image or sequence generation, such as machine translation, text summarization, and image captioning. + +!split +===== Reinforcement Learning ===== + +Reinforcement learning is a sub-area of machine learning. It studies +how agents take actions based on trial and error, so as to maximize +some notion of cumulative reward in a dynamic system or +environment. Due to its generality, the problem has also been studied +in many other disciplines, such as game theory, control theory, +operations research, information theory, multi-agent systems, swarm +intelligence, statistics, and genetic algorithms. + +In March 2016, AlphaGo, a computer program that plays the board game +Go, beat Lee Sedol in a five-game match. This was the first time a +computer Go program had beaten a 9-dan (highest rank) professional +without handicaps. AlphaGo is based on deep convolutional neural +networks and reinforcement learning. AlphaGo’s victory was a major +milestone in artificial intelligence and it has also made +reinforcement learning a hot research area in the field of machine +learning. + +!split +===== Transfer learning ===== + +The goal of transfer learning is to transfer the model or knowledge +obtained from a source task to the target task, in order to resolve +the issues of insufficient training data in the target task. The +rationality of doing so lies in that usually the source and target +tasks have inter-correlations, and therefore either the features, +samples, or models in the source task might provide useful information +for us to better solve the target task. Transfer learning is a hot +research topic in recent years, with many problems still waiting to be +solved in this space. + + +!split +===== Adversarial learning ===== + +The conventional deep generative model has a potential problem: the +model tends to generate extreme instances to maximize the +probabilistic likelihood, which will hurt its performance. Adversarial +learning utilizes the adversarial behaviors (e.g., generating +adversarial instances or training an adversarial model) to enhance the +robustness of the model and improve the quality of the generated +data. In recent years, one of the most promising unsupervised learning +technologies, generative adversarial networks (GAN), has already been +successfully applied to image, speech, and text. + +!split +===== Dual learning ===== + +Dual learning is a new learning paradigm, the basic idea of which is +to use the primal-dual structure between machine learning tasks to +obtain effective feedback/regularization, and guide and strengthen the +learning process, thus reducing the requirement of large-scale labeled +data for deep learning. The idea of dual learning has been applied to +many problems in machine learning, including machine translation, +image style conversion, question answering and generation, image +classification and generation, text classification and generation, +image-to-text, and text-to-image. + +!split +===== Distributed machine learning ===== + +Distributed computation will speed up machine learning algorithms, +significantly improve their efficiency, and thus enlarge their +application. When distributed meets machine learning, more than just +implementing the machine learning algorithms in parallel is required. + + +!split +===== Meta learning ===== + +Meta learning is an emerging research direction in machine +learning. Roughly speaking, meta learning concerns learning how to +learn, and focuses on the understanding and adaptation of the learning +itself, instead of just completing a specific learning task. That is, +a meta learner needs to be able to evaluate its own learning methods +and adjust its own learning methods according to specific learning +tasks. + +!split +===== The Challenges Facing Machine Learning ===== + +While there has been much progress in machine learning, there are also challenges. + +For example, the mainstream machine learning technologies are +black-box approaches, making us concerned about their potential +risks. To tackle this challenge, we may want to make machine learning +more explainable and controllable. As another example, the +computational complexity of machine learning algorithms is usually +very high and we may want to invent lightweight algorithms or +implementations. Furthermore, in many domains such as physics, +chemistry, biology, and social sciences, people usually seek elegantly +simple equations (e.g., the Schrödinger equation) to uncover the +underlying laws behind various phenomena. In the field of machine +learning, can we reveal simple laws instead of designing more complex +models for data fitting? Although there are many challenges, we are +still very optimistic about the future of machine learning. As we look +forward to the future, here are what we think the research hotspots in +the next ten years will be. + + +!split +===== Explainable machine learning ===== + +Machine learning, especially deep learning, evolves rapidly. The +ability gap between machine and human on many complex cognitive tasks +becomes narrower and narrower. However, we are still in the very early +stage in terms of explaining why those effective models work and how +they work. + +What is missing: the gap between correlation and causation Most +machine learning techniques, especially the statistical ones, depend +highly on data correlation to make predictions and analyses. In +contrast, rational humans tend to reply on clear and trustworthy +causality relations obtained via logical reasoning on real and clear +facts. It is one of the core goals of explainable machine learning to +transition from solving problems by data correlation to solving +problems by logical reasoning. + +!split +===== Quantum machine learning ===== + +Quantum machine learning is an emerging interdisciplinary research +area at the intersection of quantum computing and machine learning. + +Quantum computers use effects such as quantum coherence and quantum +entanglement to process information, which is fundamentally different +from classical computers. Quantum algorithms have surpassed the best +classical algorithms in several problems (e.g., searching for an +unsorted database, inverting a sparse matrix), which we call quantum +acceleration. + +When quantum computing meets machine learning, it can be a mutually +beneficial and reinforcing process, as it allows us to take advantage +of quantum computing to improve the performance of classical machine +learning algorithms. In addition, we can also use the machine learning +algorithms (on classic computers) to analyze and improve quantum +computing systems. + + +!split +===== Quantum machine learning algorithms based on linear algebra ===== + +Many quantum machine learning algorithms are based on variants of +quantum algorithms for solving linear equations, which can efficiently +solve N-variable linear equations with complexity of O(log2 N) under +certain conditions. The quantum matrix inversion algorithm can +accelerate many machine learning methods, such as least square linear +regression, least square version of support vector machine, Gaussian +process, and more. The training of these algorithms can be simplified +to solve linear equations. The key bottleneck of this type of quantum +machine learning algorithms is data input—that is, how to initialize +the quantum system with the entire data set. Although efficient +data-input algorithms exist for certain situations, how to efficiently +input data into a quantum system is as yet unknown for most cases. + +!split +===== Quantum reinforcement learning ===== + +In quantum reinforcement learning, a quantum agent interacts with the +classical environment to obtain rewards from the environment, so as to +adjust and improve its behavioral strategies. In some cases, it +achieves quantum acceleration by the quantum processing capabilities +of the agent or the possibility of exploring the environment through +quantum superposition. Such algorithms have been proposed in +superconducting circuits and systems of trapped ions. + +!split +===== Quantum deep learning ===== + +Dedicated quantum information processors, such as quantum annealers +and programmable photonic circuits, are well suited for building deep +quantum networks. The simplest deep quantum network is the Boltzmann +machine. The classical Boltzmann machine consists of bits with tunable +interactions and is trained by adjusting the interaction of these bits +so that the distribution of its expression conforms to the statistics +of the data. To quantize the Boltzmann machine, the neural network can +simply be represented as a set of interacting quantum spins that +correspond to an adjustable Ising model. Then, by initializing the +input neurons in the Boltzmann machine to a fixed state and allowing +the system to heat up, we can read out the output qubits to get the +result. + + +!split +===== Social machine learning ===== + +Machine learning aims to imitate how humans +learn. While we have developed successful machine learning algorithms, +until now we have ignored one important fact: humans are social. Each +of us is one part of the total society and it is difficult for us to +live, learn, and improve ourselves, alone and isolated. Therefore, we +should design machines with social properties. Can we let machines +evolve by imitating human society so as to achieve more effective, +intelligent, interpretable “social machine learning”? + +And much more. + +!split +===== The last words? ===== + +Early computer scientist Alan Kay said, _The best way to predict the +future is to create it_. Therefore, all machine learning +practitioners, whether scholars or engineers, professors or students, +need to work together to advance these important research +topics. Together, we will not just predict the future, but create it. + + + + !split ===== Best wishes to you all and thanks so much for your heroic efforts this semester ===== FIGURE: [figures/Nebbdyr2.png, width=500 frac=0.6] - - - - -
Additional courses of interest
Hot Topics Now
+ +-
+
+In the past few decades, many different architectures of deep neural +networks have been proposed, such as + +
-
+
+ +Reinforcement Learning
+ ++Reinforcement learning is a sub-area of machine learning. It studies +how agents take actions based on trial and error, so as to maximize +some notion of cumulative reward in a dynamic system or +environment. Due to its generality, the problem has also been studied +in many other disciplines, such as game theory, control theory, +operations research, information theory, multi-agent systems, swarm +intelligence, statistics, and genetic algorithms. + +
+In March 2016, AlphaGo, a computer program that plays the board game +Go, beat Lee Sedol in a five-game match. This was the first time a +computer Go program had beaten a 9-dan (highest rank) professional +without handicaps. AlphaGo is based on deep convolutional neural +networks and reinforcement learning. AlphaGo’s victory was a major +milestone in artificial intelligence and it has also made +reinforcement learning a hot research area in the field of machine +learning. + +
+
+ +Transfer learning
+ ++The goal of transfer learning is to transfer the model or knowledge +obtained from a source task to the target task, in order to resolve +the issues of insufficient training data in the target task. The +rationality of doing so lies in that usually the source and target +tasks have inter-correlations, and therefore either the features, +samples, or models in the source task might provide useful information +for us to better solve the target task. Transfer learning is a hot +research topic in recent years, with many problems still waiting to be +solved in this space. + +
+
+ +Adversarial learning
+ ++The conventional deep generative model has a potential problem: the +model tends to generate extreme instances to maximize the +probabilistic likelihood, which will hurt its performance. Adversarial +learning utilizes the adversarial behaviors (e.g., generating +adversarial instances or training an adversarial model) to enhance the +robustness of the model and improve the quality of the generated +data. In recent years, one of the most promising unsupervised learning +technologies, generative adversarial networks (GAN), has already been +successfully applied to image, speech, and text. + +
+
+ +Dual learning
+ ++Dual learning is a new learning paradigm, the basic idea of which is +to use the primal-dual structure between machine learning tasks to +obtain effective feedback/regularization, and guide and strengthen the +learning process, thus reducing the requirement of large-scale labeled +data for deep learning. The idea of dual learning has been applied to +many problems in machine learning, including machine translation, +image style conversion, question answering and generation, image +classification and generation, text classification and generation, +image-to-text, and text-to-image. + +
+
+ +Distributed machine learning
+ ++Distributed computation will speed up machine learning algorithms, +significantly improve their efficiency, and thus enlarge their +application. When distributed meets machine learning, more than just +implementing the machine learning algorithms in parallel is required. + +
+
+ +Meta learning
+ ++Meta learning is an emerging research direction in machine +learning. Roughly speaking, meta learning concerns learning how to +learn, and focuses on the understanding and adaptation of the learning +itself, instead of just completing a specific learning task. That is, +a meta learner needs to be able to evaluate its own learning methods +and adjust its own learning methods according to specific learning +tasks. + +
+
+ +The Challenges Facing Machine Learning
+ ++While there has been much progress in machine learning, there are also challenges. + +
+For example, the mainstream machine learning technologies are +black-box approaches, making us concerned about their potential +risks. To tackle this challenge, we may want to make machine learning +more explainable and controllable. As another example, the +computational complexity of machine learning algorithms is usually +very high and we may want to invent lightweight algorithms or +implementations. Furthermore, in many domains such as physics, +chemistry, biology, and social sciences, people usually seek elegantly +simple equations (e.g., the Schrödinger equation) to uncover the +underlying laws behind various phenomena. In the field of machine +learning, can we reveal simple laws instead of designing more complex +models for data fitting? Although there are many challenges, we are +still very optimistic about the future of machine learning. As we look +forward to the future, here are what we think the research hotspots in +the next ten years will be. + +
+
+ +Explainable machine learning
+ ++Machine learning, especially deep learning, evolves rapidly. The +ability gap between machine and human on many complex cognitive tasks +becomes narrower and narrower. However, we are still in the very early +stage in terms of explaining why those effective models work and how +they work. + +
+What is missing: the gap between correlation and causation Most +machine learning techniques, especially the statistical ones, depend +highly on data correlation to make predictions and analyses. In +contrast, rational humans tend to reply on clear and trustworthy +causality relations obtained via logical reasoning on real and clear +facts. It is one of the core goals of explainable machine learning to +transition from solving problems by data correlation to solving +problems by logical reasoning. + +
+
+ +Quantum machine learning
+ ++Quantum machine learning is an emerging interdisciplinary research +area at the intersection of quantum computing and machine learning. + +
+Quantum computers use effects such as quantum coherence and quantum +entanglement to process information, which is fundamentally different +from classical computers. Quantum algorithms have surpassed the best +classical algorithms in several problems (e.g., searching for an +unsorted database, inverting a sparse matrix), which we call quantum +acceleration. + +
+When quantum computing meets machine learning, it can be a mutually +beneficial and reinforcing process, as it allows us to take advantage +of quantum computing to improve the performance of classical machine +learning algorithms. In addition, we can also use the machine learning +algorithms (on classic computers) to analyze and improve quantum +computing systems. + +
+
+ +Quantum machine learning algorithms based on linear algebra
+ ++Many quantum machine learning algorithms are based on variants of +quantum algorithms for solving linear equations, which can efficiently +solve N-variable linear equations with complexity of O(log2 N) under +certain conditions. The quantum matrix inversion algorithm can +accelerate many machine learning methods, such as least square linear +regression, least square version of support vector machine, Gaussian +process, and more. The training of these algorithms can be simplified +to solve linear equations. The key bottleneck of this type of quantum +machine learning algorithms is data input—that is, how to initialize +the quantum system with the entire data set. Although efficient +data-input algorithms exist for certain situations, how to efficiently +input data into a quantum system is as yet unknown for most cases. + +
+
+ +Quantum reinforcement learning
+ ++In quantum reinforcement learning, a quantum agent interacts with the +classical environment to obtain rewards from the environment, so as to +adjust and improve its behavioral strategies. In some cases, it +achieves quantum acceleration by the quantum processing capabilities +of the agent or the possibility of exploring the environment through +quantum superposition. Such algorithms have been proposed in +superconducting circuits and systems of trapped ions. + +
+
+ +Quantum deep learning
+ ++Dedicated quantum information processors, such as quantum annealers +and programmable photonic circuits, are well suited for building deep +quantum networks. The simplest deep quantum network is the Boltzmann +machine. The classical Boltzmann machine consists of bits with tunable +interactions and is trained by adjusting the interaction of these bits +so that the distribution of its expression conforms to the statistics +of the data. To quantize the Boltzmann machine, the neural network can +simply be represented as a set of interacting quantum spins that +correspond to an adjustable Ising model. Then, by initializing the +input neurons in the Boltzmann machine to a fixed state and allowing +the system to heat up, we can read out the output qubits to get the +result. + +
+
+ +Social machine learning
+ ++Machine learning aims to imitate how humans +learn. While we have developed successful machine learning algorithms, +until now we have ignored one important fact: humans are social. Each +of us is one part of the total society and it is difficult for us to +live, learn, and improve ourselves, alone and isolated. Therefore, we +should design machines with social properties. Can we let machines +evolve by imitating human society so as to achieve more effective, +intelligent, interpretable “social machine learning”? + +
+And much more. + +
+
+ +The last words?
+ ++Early computer scientist Alan Kay said, The best way to predict the +future is to create it. Therefore, all machine learning +practitioners, whether scholars or engineers, professors or students, +need to work together to advance these important research +topics. Together, we will not just predict the future, but create it. + +
+
+ +Best wishes to you all and thanks so much for your heroic efforts this semester
-- diff --git a/doc/pub/summary/html/summary.html b/doc/pub/summary/html/summary.html index 89656495a..e2a75ab16 100644 --- a/doc/pub/summary/html/summary.html +++ b/doc/pub/summary/html/summary.html @@ -57,26 +57,43 @@ div { text-align: justify; text-justify: inter-word; } '___sec5'), ('Perspective on Machine Learning', 2, None, '___sec6'), ('Machine Learning Research', 2, None, '___sec7'), - ('Hot Topics Now', 2, None, '___sec8'), - ('Starting your Machine Learning Project', 2, None, '___sec9'), - ('Choose a Model and Algorithm', 2, None, '___sec10'), - ('Preparing Your Data', 2, None, '___sec11'), + ('Starting your Machine Learning Project', 2, None, '___sec8'), + ('Choose a Model and Algorithm', 2, None, '___sec9'), + ('Preparing Your Data', 2, None, '___sec10'), ('Which Activation and Weights to Choose in Neural Networks', 2, None, - '___sec12'), - ('Optimization Methods and Hyperparameters', 2, None, '___sec13'), - ('Resampling', 2, None, '___sec14'), + '___sec11'), + ('Optimization Methods and Hyperparameters', 2, None, '___sec12'), + ('Resampling', 2, None, '___sec13'), ('Other courses on Data science and Machine Learning at UiO', 2, None, - '___sec15'), - ('Additional courses of interest', 2, None, '___sec16'), + '___sec14'), + ('Additional courses of interest', 2, None, '___sec15'), + ('Hot Topics Now', 2, None, '___sec16'), + ('Reinforcement Learning', 2, None, '___sec17'), + ('Transfer learning', 2, None, '___sec18'), + ('Adversarial learning', 2, None, '___sec19'), + ('Dual learning', 2, None, '___sec20'), + ('Distributed machine learning', 2, None, '___sec21'), + ('Meta learning', 2, None, '___sec22'), + ('The Challenges Facing Machine Learning', 2, None, '___sec23'), + ('Explainable machine learning', 2, None, '___sec24'), + ('Quantum machine learning', 2, None, '___sec25'), + ('Quantum machine learning algorithms based on linear algebra', + 2, + None, + '___sec26'), + ('Quantum reinforcement learning', 2, None, '___sec27'), + ('Quantum deep learning', 2, None, '___sec28'), + ('Social machine learning', 2, None, '___sec29'), + ('The last words?', 2, None, '___sec30'), ('Best wishes to you all and thanks so much for your heroic ' 'efforts this semester', 2, None, - '___sec17')]} + '___sec31')]} end of tocinfo -->
@@ -244,19 +261,7 @@ Where to find recent results:
-Hot Topics Now
- --
-
- -Starting your Machine Learning Project
+Starting your Machine Learning Project
-Choose a Model and Algorithm
+Choose a Model and Algorithm
-Preparing Your Data
+Preparing Your Data
-Which Activation and Weights to Choose in Neural Networks
+Which Activation and Weights to Choose in Neural Networks
-Optimization Methods and Hyperparameters
+Optimization Methods and Hyperparameters
-Choose a Model and Algorithm
+Choose a Model and Algorithm
-Preparing Your Data
+Preparing Your Data
-Which Activation and Weights to Choose in Neural Networks
+Which Activation and Weights to Choose in Neural Networks
-Optimization Methods and Hyperparameters
+Optimization Methods and Hyperparameters