@@ -459,7 +459,7 @@ long run. Here we have only defined the training data and our model, and have not discussed a more rigorous approach to the cost function.
-We need more rigorous criteria in defining whether we have succeed or +We need more rigorous criteria in defining whether we have succeeded or not in modeling our training data. You will be surprised to see that many scientists seldomly venture beyond this 'by the eye' approach. A standard approach for the cost function is the so-called \( \chi^2 \) diff --git a/doc/pub/How2ReadData/html/How2ReadData-reveal.html b/doc/pub/How2ReadData/html/How2ReadData-reveal.html index 7c11d1c34..1aa9ac1b8 100644 --- a/doc/pub/How2ReadData/html/How2ReadData-reveal.html +++ b/doc/pub/How2ReadData/html/How2ReadData-reveal.html @@ -415,7 +415,7 @@ function \( y \) to
$$
-y = 10x+0.01N(0,1),
+y = 10x+0.01 \times N(0,1),
$$
@@ -428,7 +428,7 @@ long run. Here we have only defined the training data and our model, and
have not discussed a more rigorous approach to the cost function.
-We need more rigorous criteria in defining whether we have succeed or +We need more rigorous criteria in defining whether we have succeeded or not in modeling our training data. You will be surprised to see that many scientists seldomly venture beyond this 'by the eye' approach. A standard approach for the cost function is the so-called \( \chi^2 \) diff --git a/doc/pub/How2ReadData/html/How2ReadData-solarized.html b/doc/pub/How2ReadData/html/How2ReadData-solarized.html index b17a9732e..8814a5e31 100644 --- a/doc/pub/How2ReadData/html/How2ReadData-solarized.html +++ b/doc/pub/How2ReadData/html/How2ReadData-solarized.html @@ -404,7 +404,7 @@ factor in front of \( x \) and the normal distribution. Try to change the function \( y \) to $$ -y = 10x+0.01N(0,1), +y = 10x+0.01 \times N(0,1), $$
@@ -416,7 +416,7 @@ long run. Here we have only defined the training data and our model, and have not discussed a more rigorous approach to the cost function.
-We need more rigorous criteria in defining whether we have succeed or +We need more rigorous criteria in defining whether we have succeeded or not in modeling our training data. You will be surprised to see that many scientists seldomly venture beyond this 'by the eye' approach. A standard approach for the cost function is the so-called \( \chi^2 \) diff --git a/doc/pub/How2ReadData/html/How2ReadData.html b/doc/pub/How2ReadData/html/How2ReadData.html index 80d7429af..6a37184b9 100644 --- a/doc/pub/How2ReadData/html/How2ReadData.html +++ b/doc/pub/How2ReadData/html/How2ReadData.html @@ -409,7 +409,7 @@ factor in front of \( x \) and the normal distribution. Try to change the function \( y \) to $$ -y = 10x+0.01N(0,1), +y = 10x+0.01 \times N(0,1), $$
@@ -421,7 +421,7 @@ long run. Here we have only defined the training data and our model, and have not discussed a more rigorous approach to the cost function.
-We need more rigorous criteria in defining whether we have succeed or +We need more rigorous criteria in defining whether we have succeeded or not in modeling our training data. You will be surprised to see that many scientists seldomly venture beyond this 'by the eye' approach. A standard approach for the cost function is the so-called \( \chi^2 \) diff --git a/doc/pub/How2ReadData/ipynb/How2ReadData.ipynb b/doc/pub/How2ReadData/ipynb/How2ReadData.ipynb index 48fa4fe11..d10084e44 100644 --- a/doc/pub/How2ReadData/ipynb/How2ReadData.ipynb +++ b/doc/pub/How2ReadData/ipynb/How2ReadData.ipynb @@ -268,7 +268,7 @@ "metadata": {}, "source": [ "$$\n", - "y = 10x+0.01N(0,1),\n", + "y = 10x+0.01 \\times N(0,1),\n", "$$" ] }, @@ -283,7 +283,7 @@ "long run. Here we have only defined the training data and our model, and \n", "have not discussed a more rigorous approach to the **cost** function.\n", "\n", - "We need more rigorous criteria in defining whether we have succeed or\n", + "We need more rigorous criteria in defining whether we have succeeded or\n", "not in modeling our training data. You will be surprised to see that\n", "many scientists seldomly venture beyond this 'by the eye' approach. A\n", "standard approach for the *cost* function is the so-called $\\chi^2$\n", diff --git a/doc/pub/How2ReadData/ipynb/ipynb-How2ReadData-src.tar.gz b/doc/pub/How2ReadData/ipynb/ipynb-How2ReadData-src.tar.gz index 532ba34d7..950467d98 100644 Binary files a/doc/pub/How2ReadData/ipynb/ipynb-How2ReadData-src.tar.gz and b/doc/pub/How2ReadData/ipynb/ipynb-How2ReadData-src.tar.gz differ diff --git a/doc/pub/How2ReadData/pdf/How2ReadData-beamer-handouts2x3.pdf b/doc/pub/How2ReadData/pdf/How2ReadData-beamer-handouts2x3.pdf index e76f26e75..181b5b427 100644 Binary files a/doc/pub/How2ReadData/pdf/How2ReadData-beamer-handouts2x3.pdf and b/doc/pub/How2ReadData/pdf/How2ReadData-beamer-handouts2x3.pdf differ diff --git a/doc/pub/How2ReadData/pdf/How2ReadData-beamer.pdf b/doc/pub/How2ReadData/pdf/How2ReadData-beamer.pdf index 28794b1c1..42e2f1dd3 100644 Binary files a/doc/pub/How2ReadData/pdf/How2ReadData-beamer.pdf and b/doc/pub/How2ReadData/pdf/How2ReadData-beamer.pdf differ diff --git a/doc/pub/How2ReadData/pdf/How2ReadData-minted.pdf b/doc/pub/How2ReadData/pdf/How2ReadData-minted.pdf index ee8c1fabd..e8e88a825 100644 Binary files a/doc/pub/How2ReadData/pdf/How2ReadData-minted.pdf and b/doc/pub/How2ReadData/pdf/How2ReadData-minted.pdf differ diff --git a/doc/pub/Introduction/html/Introduction-bs.html b/doc/pub/Introduction/html/Introduction-bs.html index ec565f021..a052cad50 100644 --- a/doc/pub/Introduction/html/Introduction-bs.html +++ b/doc/pub/Introduction/html/Introduction-bs.html @@ -6,9 +6,9 @@ Automatically generated HTML file from DocOnce source
- + -@@ -111,14 +107,11 @@ end of tocinfo -->
-
-
@@ -126,57 +119,68 @@ Statistics, data science and machine learning form important fields of research in modern science. They describe how to learn and make predictions from data, as well as allowing us to extract important correlations about physical process and the underlying laws of motion -in large data sets. The latter, big data sets, appear -frequently in essentially all disciplines, from the traditional Science, -Technology, Mathematics and Engineering fields to Life Science, Law, education research, -the Humanities and -the Social Sciences. It has become more and more common to see -research projects on big data in for example the Social -Sciences where extracting patterns from complicated survey data is one of many research directions. -Having a solid grasp of data analysis and machine learning -is thus becoming central to scientific computing in many -fields, and competences and skills within the fields of machine learning -and scientific computing are nowadays strongly requested by many -potential employers. The latter cannot be overstated, familiarity with -machine learning has almost become a prerequisite for many of the most -exciting employment opportunities, whether they are in bioinformatics, -life science, physics or finance, in the private or the public -sector. This author has had several students or met students who have -been hired recently based on their skills and competences in -scientific computing and data science, often with marginal knowledge -of machine learning. +in large data sets. The latter, big data sets, appear frequently in +essentially all disciplines, from the traditional Science, Technology, +Mathematics and Engineering fields to Life Science, Law, education +research, the Humanities and the Social Sciences. + +
+It has become more +and more common to see research projects on big data in for example +the Social Sciences where extracting patterns from complicated survey +data is one of many research directions. Having a solid grasp of data +analysis and machine learning is thus becoming central to scientific +computing in many fields, and competences and skills within the fields +of machine learning and scientific computing are nowadays strongly +requested by many potential employers. The latter cannot be +overstated, familiarity with machine learning has almost become a +prerequisite for many of the most exciting employment opportunities, +whether they are in bioinformatics, life science, physics or finance, +in the private or the public sector. This author has had several +students or met students who have been hired recently based on their +skills and competences in scientific computing and data science, often +with marginal knowledge of machine learning.
Machine learning is a subfield of computer science, and is closely related to computational statistics. It evolved from the study of pattern recognition in artificial intelligence (AI) research, and has made contributions to AI tasks like computer vision, natural language -processing and speech recognition. -Machine learning represents the -science of giving computers the ability to learn without being -explicitly programmed. The idea is that there exist generic -algorithms which can be used to find patterns in a broad class of data -sets without having to write code specifically for each problem. The -algorithm will build its own logic based on the data. +processing and speech recognition. Many of the methods we will study are also +strongly rooted in basic mathematics and physics research.
-Machine learning is an extremely rich field, in spite of its young age. The -increases we have seen during the last three decades in computational -capabilities have been followed by developments of methods and -techniques for analyzing and handling large date sets, relying heavily -on statistics, computer science and mathematics. The field is rather -new and developing rapidly. Popular software packages written in -Python for machine learning like Scikit-learn, Tensorflow, -PyTorch and Keras, all freely available at their respective GitHub sites, -encompass communities of developers in the thousands or more. And the number -of code developers and contributors keeps increasing. Not all the +Ideally, machine learning represents the science of giving computers +the ability to learn without being explicitly programmed. The idea is +that there exist generic algorithms which can be used to find patterns +in a broad class of data sets without having to write code +specifically for each problem. The algorithm will build its own logic +based on the data. You should however always keep in mind that +machines and algorithms are to a large extent developed by humans. The +insights and knowledge we have about a specific system, play a central +role when we develop a specific machine learning algorithm. + +
+Machine learning is an extremely rich field, in spite of its young +age. The increases we have seen during the last three decades in +computational capabilities have been followed by developments of +methods and techniques for analyzing and handling large date sets, +relying heavily on statistics, computer science and mathematics. The +field is rather new and developing rapidly. Popular software packages +written in Python for machine learning like +Scikit-learn, +Tensorflow, +PyTorch and Keras, all +freely available at their respective GitHub sites, encompass +communities of developers in the thousands or more. And the number of +code developers and contributors keeps increasing. Not all the algorithms and methods can be given a rigorous mathematical -justification, opening up thereby large rooms for experimenting -and trial and error and thereby exciting new developments. -However, a solid command of linear algebra, multivariate theory, -probability theory, statistical data analysis, -understanding errors and Monte Carlo methods are central elements in a proper understanding of many of -algorithms and methods we will discuss. +justification, opening up thereby large rooms for experimenting and +trial and error and thereby exciting new developments. However, a +solid command of linear algebra, multivariate theory, probability +theory, statistical data analysis, understanding errors and Monte +Carlo methods are central elements in a proper understanding of many +of algorithms and methods we will discuss.
@@ -184,7 +188,7 @@ algorithms and methods we will discuss.
-These lectures aim at giving you an overview of central aspects of +These setsof lectures aim at giving you an overview of central aspects of statistical data analysis as well as some of the central algorithms used in machine learning. We will introduce a variety of central algorithms and methods essential for studies of data analysis and @@ -235,23 +239,22 @@ and jackknife methods. The second part of the material covers several algorithms used in machine learning. -
- -
-The approaches to machine learning are many, but are often split into two main categories. -In supervised learning we know the answer to a problem, -and let the computer deduce the logic behind it. On the other hand, unsupervised learning -is a method for finding patterns and relationship in data sets without any prior knowledge of the system. -Some authours also operate with a third category, namely reinforcement learning. This is a paradigm -of learning inspired by behavioral psychology, where learning is achieved by trial-and-error, +The approaches to machine learning are many, but are often split into +two main categories. In supervised learning we know the answer to a +problem, and let the computer deduce the logic behind it. On the other +hand, unsupervised learning is a method for finding patterns and +relationship in data sets without any prior knowledge of the system. +Some authours also operate with a third category, namely +reinforcement learning. This is a paradigm of learning inspired by +behavioral psychology, where learning is achieved by trial-and-error, solely from rewards and punishment.
-Another way to categorize machine learning tasks is to consider the desired output of a system. -Some of the most common tasks are: +Another way to categorize machine learning tasks is to consider the +desired output of a system. Some of the most common tasks are:
+The last ingredient is a so-called cost +function which allows us to present an estimate on how good our model +is in reproducing the data it is supposed to train.
Here we will build our machine learning approach on elements of the @@ -279,28 +286,161 @@ machine learning algorithms
- - -
- - -
- - -
+Many of the examples we discuss in this series of lectures come with +existing data files or provide code examples which produce the data to +be analyzed. Most of the applications we will discuss deal with +small data sets (less than a terabyte of information) and can easily +be analyzed and tested on standard off the shelf laptops you find in general +grocery stores. + +
+In most of the cases we will study, we will either generate the data +to analyze ourselves (both for supervised learning and unsupervised +learning) or we will recur again and again to data present in say +scikit-learn or tensorflow. Many of the examples we end up +dealing with are from a privacy and data protection point of view, +rather inoccuous and boring results of numerical +calculations. However, this does not hinder us from developing a sound +ethical attitude to the data we use, how we analyze the data and how +we handle the data. + +
+The most immediate and simplest possible ethical aspects deal with our +approach to the scientific process. Nowadays, with version control +software like Git and various online +repositories like Github, +Gitlab etc, we can easily make our codes +and data sets we have used, freely and easily accessible to a wider +community. This helps us almost automagically in making our science +reproducible. The large open-source development communities involved +in say Scikit-learn, +Tensorflow, +PyTorch and Keras, are +all excellent examples of this. The codes can be tested and improved +upon continuosly, helping thereby our scientific community at large in +developing data analysis and machine learning tools. It is much +easier today to gain traction and acceptance for making your science +reproducible. From a societal stand, this is an important element +since many of the developers are employees of large public institutions like +universities and research labs. Our taxpayer do deserve to get +something back for their bucks. + +
+However, this more mechanical aspect of the ethics of science (in +particular the reproducibility of scientific results) is something +which is obvious and everybody should do as part of the dialectics of +science. The fact that many scientists are not willing to share their codes or +data is detrimental to the scientific discourse. + +
+Before we proceed, we should add a disclaimer. Even though +we may dream of computers developing some kind of higher learning +capabilities, at the end (even if the artificial intelligence +community keeps touting our ears full of fancy futuristic avenues), it is we +who end up constructing and instructing, via various algorithms, the +computers. Self-driving cars for example, rely on sofisticated +programs which take into account all possible situations a car can +encounter. In addition, extensive usage of training datas from GPS +information, maps etc, are typically fed into the software for +self-driving cars. Adding to this various sensors and cameras that +feed information to the programs, there are zillions of ethical issues +which arise from this. + +
+For self-driving cars, where basically many of the standard machine +learning algorithms discussed here enter into the codes, at a certain +stage we have to make choices. Yes, we , the lads and lasses who wrote +a program for a specific brand of a self-driving car. As an example, +a most carmakers have as their utmost priority the security of the +driver and the accompanying passengers. A famous carmaker, which is +one of the leaders in the market of self-driving cars, had if +statements of the following type: suppose there are two obstacles in +front of you and you cannot avoid to collide with one of them. One of +the obstacles is a monstertruck while the other one is a kindergarten +class trying to cross the road. The self-driving car algo would then +opt for the hitting the small folks instead of the monstertruck, since +the likelihood of surving a collision with our future citizens, is +much higher. + +
+This brings us leads then to serious ethical aspects. Why should we +opt for such an option? Who decides and who is entitled to make such +choices? Keep in mind that many of the algorithms you will about in +this series of lectures or hear about later, are indeed based on +simple programming instructions. And you are very likely to be one of +the people who may end up writing such a code. Thus, developing a +sound ethical attitude to what we do, an approach well beyond the +simple mechanistic one of making our science available and +reproducible, is much needed. The example of the self-driving cars is +just one of infinitely many cases where we have to make choices. When +you analyze data on economic inequalities, who guarantees that you are +not weighting some data in a particular way, perhaps because you dearly want a +specific conclusion which may support your political views? + +
+We do not have the answers here, but we want you think over these +topics in a more overarching way. A statistical data analysis with +its dry numbers and graphs meant to guide the eye, do not necessarily +reflect the truth, whatever that is. As a scientist, and after a +university education, you are supposedly a better citizen, with an +improved critical view and understanding of the scientific method, and +perhaps some deeper understandings of the ethics of science at +large. Use these insights. Be a critical citizen. You owe it to our +societies. + +
+To do: Add references and acknowledgements diff --git a/doc/pub/Introduction/html/Introduction-reveal.html b/doc/pub/Introduction/html/Introduction-reveal.html index a34392da4..8565b2273 100644 --- a/doc/pub/Introduction/html/Introduction-reveal.html +++ b/doc/pub/Introduction/html/Introduction-reveal.html @@ -3,9 +3,9 @@ - + -
@@ -132,8 +132,79 @@ td.padding {
-
+Statistics, data science and machine learning form important fields of +research in modern science. They describe how to learn and make +predictions from data, as well as allowing us to extract important +correlations about physical process and the underlying laws of motion +in large data sets. The latter, big data sets, appear frequently in +essentially all disciplines, from the traditional Science, Technology, +Mathematics and Engineering fields to Life Science, Law, education +research, the Humanities and the Social Sciences. + +
+It has become more +and more common to see research projects on big data in for example +the Social Sciences where extracting patterns from complicated survey +data is one of many research directions. Having a solid grasp of data +analysis and machine learning is thus becoming central to scientific +computing in many fields, and competences and skills within the fields +of machine learning and scientific computing are nowadays strongly +requested by many potential employers. The latter cannot be +overstated, familiarity with machine learning has almost become a +prerequisite for many of the most exciting employment opportunities, +whether they are in bioinformatics, life science, physics or finance, +in the private or the public sector. This author has had several +students or met students who have been hired recently based on their +skills and competences in scientific computing and data science, often +with marginal knowledge of machine learning. + +
+Machine learning is a subfield of computer science, and is closely +related to computational statistics. It evolved from the study of +pattern recognition in artificial intelligence (AI) research, and has +made contributions to AI tasks like computer vision, natural language +processing and speech recognition. Many of the methods we will study are also +strongly rooted in basic mathematics and physics research. + +
+Ideally, machine learning represents the science of giving computers +the ability to learn without being explicitly programmed. The idea is +that there exist generic algorithms which can be used to find patterns +in a broad class of data sets without having to write code +specifically for each problem. The algorithm will build its own logic +based on the data. You should however always keep in mind that +machines and algorithms are to a large extent developed by humans. The +insights and knowledge we have about a specific system, play a central +role when we develop a specific machine learning algorithm. + +
+Machine learning is an extremely rich field, in spite of its young +age. The increases we have seen during the last three decades in +computational capabilities have been followed by developments of +methods and techniques for analyzing and handling large date sets, +relying heavily on statistics, computer science and mathematics. The +field is rather new and developing rapidly. Popular software packages +written in Python for machine learning like +Scikit-learn, +Tensorflow, +PyTorch and Keras, all +freely available at their respective GitHub sites, encompass +communities of developers in the thousands or more. And the number of +code developers and contributors keeps increasing. Not all the +algorithms and methods can be given a rigorous mathematical +justification, opening up thereby large rooms for experimenting and +trial and error and thereby exciting new developments. However, a +solid command of linear algebra, multivariate theory, probability +theory, statistical data analysis, understanding errors and Monte +Carlo methods are central elements in a proper understanding of many +of algorithms and methods we will discuss. +
-Statistics, data science and machine learning form important fields of -research in modern science. They describe how to learn and make -predictions from data, as well as allowing us to extract important -correlations about physical process and the underlying laws of motion -in large data sets. The latter, big data sets, appear -frequently in essentially all disciplines, from the traditional Science, -Technology, Mathematics and Engineering fields to Life Science, Law, education research, -the Humanities and -the Social Sciences. It has become more and more common to see -research projects on big data in for example the Social -Sciences where extracting patterns from complicated survey data is one of many research directions. -Having a solid grasp of data analysis and machine learning -is thus becoming central to scientific computing in many -fields, and competences and skills within the fields of machine learning -and scientific computing are nowadays strongly requested by many -potential employers. The latter cannot be overstated, familiarity with -machine learning has almost become a prerequisite for many of the most -exciting employment opportunities, whether they are in bioinformatics, -life science, physics or finance, in the private or the public -sector. This author has had several students or met students who have -been hired recently based on their skills and competences in -scientific computing and data science, often with marginal knowledge -of machine learning. - -
-Machine learning is a subfield of computer science, and is closely -related to computational statistics. It evolved from the study of -pattern recognition in artificial intelligence (AI) research, and has -made contributions to AI tasks like computer vision, natural language -processing and speech recognition. -Machine learning represents the -science of giving computers the ability to learn without being -explicitly programmed. The idea is that there exist generic -algorithms which can be used to find patterns in a broad class of data -sets without having to write code specifically for each problem. The -algorithm will build its own logic based on the data. - -
-Machine learning is an extremely rich field, in spite of its young age. The -increases we have seen during the last three decades in computational -capabilities have been followed by developments of methods and -techniques for analyzing and handling large date sets, relying heavily -on statistics, computer science and mathematics. The field is rather -new and developing rapidly. Popular software packages written in -Python for machine learning like Scikit-learn, Tensorflow, -PyTorch and Keras, all freely available at their respective GitHub sites, -encompass communities of developers in the thousands or more. And the number -of code developers and contributors keeps increasing. Not all the -algorithms and methods can be given a rigorous mathematical -justification, opening up thereby large rooms for experimenting -and trial and error and thereby exciting new developments. -However, a solid command of linear algebra, multivariate theory, -probability theory, statistical data analysis, -understanding errors and Monte Carlo methods are central elements in a proper understanding of many of -algorithms and methods we will discuss. -
-These lectures aim at giving you an overview of central aspects of +These setsof lectures aim at giving you an overview of central aspects of statistical data analysis as well as some of the central algorithms used in machine learning. We will introduce a variety of central algorithms and methods essential for studies of data analysis and @@ -259,24 +268,23 @@ and jackknife methods.
The second part of the material covers several algorithms used in machine learning. -
-The approaches to machine learning are many, but are often split into two main categories. -In supervised learning we know the answer to a problem, -and let the computer deduce the logic behind it. On the other hand, unsupervised learning -is a method for finding patterns and relationship in data sets without any prior knowledge of the system. -Some authours also operate with a third category, namely reinforcement learning. This is a paradigm -of learning inspired by behavioral psychology, where learning is achieved by trial-and-error, +The approaches to machine learning are many, but are often split into +two main categories. In supervised learning we know the answer to a +problem, and let the computer deduce the logic behind it. On the other +hand, unsupervised learning is a method for finding patterns and +relationship in data sets without any prior knowledge of the system. +Some authours also operate with a third category, namely +reinforcement learning. This is a paradigm of learning inspired by +behavioral psychology, where learning is achieved by trial-and-error, solely from rewards and punishment.
-Another way to categorize machine learning tasks is to consider the desired output of a system. -Some of the most common tasks are: +Another way to categorize machine learning tasks is to consider the +desired output of a system. Some of the most common tasks are:
+The last ingredient is a so-called cost +function which allows us to present an estimate on how good our model +is in reproducing the data it is supposed to train.
Here we will build our machine learning approach on elements of the @@ -308,27 +320,161 @@ machine learning algorithms
+Python plays nowadays a central role in the development of machine +learning techniques and tools for data analysis. In particular, seen +the wealth of machine learning and data analysis packages written in +Python, easy to use libraries with immediate visualization(and not the +least impressive galleries of existing example), the popularity of the +Jupyter notebook framework with the possibility to run R codes or +compiled programs written in C++, and much more made our choice of +programming language for this series of lectures of easy. However, +since the focus here is not only on using existing Python tools such +as scikit-learn or tensorflow, but also on developing your own +algorithms and codes, we will as far as possible present many of these +algorithms eithers a Python codes or C++ codes. Finally, we will, as +far as possible keep parallel versions of the data analysis and +machine larning programming aspects in R as +well. R is a language and environment +for statistical computing and graphics which is widely used in +statistics and mathematics applications. +
+The reason we also focus on compiled languages like C++ (or
+Fortran), is that Python is still notoriously slow when we do not
+utilize highly streamlined computational libraries like
+Lapack or other numerical libraries
+written in compiled languages (many of these libraries are written in
+Fortran). Although a project like Numba
+holds great promise for speeding up the unrolling of lengthy loops, C+
+and Fortran are presently still the performance winners. Numba gives
+you potentially the power to speed up your applications with high
+performance functions written directly in Python. In particular,
+array-oriented and math-heavy Python code can achieve similar
+performance to C, C++ and Fortran. However, even with these speed-ups,
+for codes involving heavy Markov Chain Monte Carlo analyses and
+optimizations of cost functions, C++/C or Fortran codes tend to
+outperform Python codes.
-Choice of programming language
-
+Presently thus, the community tends to let +code written in C++/C or Fortran do the heavy duty numerical +number crunching and leave the post-analysis of the data to the above +mentioned Python modules or software packages. However, with the developments taking place in for example the Python community, and seen +the changes during the last decade, the above situation may change swiftly in the not too distant future. +
+Many of the examples we discuss in this series of lectures come with
+existing data files or provide code examples which produce the data to
+be analyzed. Most of the applications we will discuss deal with
+small data sets (less than a terabyte of information) and can easily
+be analyzed and tested on standard off the shelf laptops you find in general
+grocery stores.
-Data handling, machine learning and ethical aspects
-
+In most of the cases we will study, we will either generate the data
+to analyze ourselves (both for supervised learning and unsupervised
+learning) or we will recur again and again to data present in say
+scikit-learn or tensorflow. Many of the examples we end up
+dealing with are from a privacy and data protection point of view,
+rather inoccuous and boring results of numerical
+calculations. However, this does not hinder us from developing a sound
+ethical attitude to the data we use, how we analyze the data and how
+we handle the data.
-
+The most immediate and simplest possible ethical aspects deal with our
+approach to the scientific process. Nowadays, with version control
+software like Git and various online
+repositories like Github,
+Gitlab etc, we can easily make our codes
+and data sets we have used, freely and easily accessible to a wider
+community. This helps us almost automagically in making our science
+reproducible. The large open-source development communities involved
+in say Scikit-learn,
+Tensorflow,
+PyTorch and Keras, are
+all excellent examples of this. The codes can be tested and improved
+upon continuosly, helping thereby our scientific community at large in
+developing data analysis and machine learning tools. It is much
+easier today to gain traction and acceptance for making your science
+reproducible. From a societal stand, this is an important element
+since many of the developers are employees of large public institutions like
+universities and research labs. Our taxpayer do deserve to get
+something back for their bucks.
+
+
+However, this more mechanical aspect of the ethics of science (in
+particular the reproducibility of scientific results) is something
+which is obvious and everybody should do as part of the dialectics of
+science. The fact that many scientists are not willing to share their codes or
+data is detrimental to the scientific discourse.
+
+
+Before we proceed, we should add a disclaimer. Even though
+we may dream of computers developing some kind of higher learning
+capabilities, at the end (even if the artificial intelligence
+community keeps touting our ears full of fancy futuristic avenues), it is we
+who end up constructing and instructing, via various algorithms, the
+computers. Self-driving cars for example, rely on sofisticated
+programs which take into account all possible situations a car can
+encounter. In addition, extensive usage of training datas from GPS
+information, maps etc, are typically fed into the software for
+self-driving cars. Adding to this various sensors and cameras that
+feed information to the programs, there are zillions of ethical issues
+which arise from this.
+
+
+For self-driving cars, where basically many of the standard machine
+learning algorithms discussed here enter into the codes, at a certain
+stage we have to make choices. Yes, we , the lads and lasses who wrote
+a program for a specific brand of a self-driving car. As an example,
+a most carmakers have as their utmost priority the security of the
+driver and the accompanying passengers. A famous carmaker, which is
+one of the leaders in the market of self-driving cars, had if
+statements of the following type: suppose there are two obstacles in
+front of you and you cannot avoid to collide with one of them. One of
+the obstacles is a monstertruck while the other one is a kindergarten
+class trying to cross the road. The self-driving car algo would then
+opt for the hitting the small folks instead of the monstertruck, since
+the likelihood of surving a collision with our future citizens, is
+much higher.
+
+
+This brings us leads then to serious ethical aspects. Why should we
+opt for such an option? Who decides and who is entitled to make such
+choices? Keep in mind that many of the algorithms you will about in
+this series of lectures or hear about later, are indeed based on
+simple programming instructions. And you are very likely to be one of
+the people who may end up writing such a code. Thus, developing a
+sound ethical attitude to what we do, an approach well beyond the
+simple mechanistic one of making our science available and
+reproducible, is much needed. The example of the self-driving cars is
+just one of infinitely many cases where we have to make choices. When
+you analyze data on economic inequalities, who guarantees that you are
+not weighting some data in a particular way, perhaps because you dearly want a
+specific conclusion which may support your political views?
+
+
+We do not have the answers here, but we want you think over these
+topics in a more overarching way. A statistical data analysis with
+its dry numbers and graphs meant to guide the eye, do not necessarily
+reflect the truth, whatever that is. As a scientist, and after a
+university education, you are supposedly a better citizen, with an
+improved critical view and understanding of the scientific method, and
+perhaps some deeper understandings of the ethics of science at
+large. Use these insights. Be a critical citizen. You owe it to our
+societies.
+
+
+To do: Add references and acknowledgements
Acknowledgements
+
@@ -69,10 +67,8 @@ end of tocinfo -->
-
-
+It has become more +and more common to see research projects on big data in for example +the Social Sciences where extracting patterns from complicated survey +data is one of many research directions. Having a solid grasp of data +analysis and machine learning is thus becoming central to scientific +computing in many fields, and competences and skills within the fields +of machine learning and scientific computing are nowadays strongly +requested by many potential employers. The latter cannot be +overstated, familiarity with machine learning has almost become a +prerequisite for many of the most exciting employment opportunities, +whether they are in bioinformatics, life science, physics or finance, +in the private or the public sector. This author has had several +students or met students who have been hired recently based on their +skills and competences in scientific computing and data science, often +with marginal knowledge of machine learning.
Machine learning is a subfield of computer science, and is closely related to computational statistics. It evolved from the study of pattern recognition in artificial intelligence (AI) research, and has made contributions to AI tasks like computer vision, natural language -processing and speech recognition. -Machine learning represents the -science of giving computers the ability to learn without being -explicitly programmed. The idea is that there exist generic -algorithms which can be used to find patterns in a broad class of data -sets without having to write code specifically for each problem. The -algorithm will build its own logic based on the data. +processing and speech recognition. Many of the methods we will study are also +strongly rooted in basic mathematics and physics research.
-Machine learning is an extremely rich field, in spite of its young age. The -increases we have seen during the last three decades in computational -capabilities have been followed by developments of methods and -techniques for analyzing and handling large date sets, relying heavily -on statistics, computer science and mathematics. The field is rather -new and developing rapidly. Popular software packages written in -Python for machine learning like Scikit-learn, Tensorflow, -PyTorch and Keras, all freely available at their respective GitHub sites, -encompass communities of developers in the thousands or more. And the number -of code developers and contributors keeps increasing. Not all the +Ideally, machine learning represents the science of giving computers +the ability to learn without being explicitly programmed. The idea is +that there exist generic algorithms which can be used to find patterns +in a broad class of data sets without having to write code +specifically for each problem. The algorithm will build its own logic +based on the data. You should however always keep in mind that +machines and algorithms are to a large extent developed by humans. The +insights and knowledge we have about a specific system, play a central +role when we develop a specific machine learning algorithm. + +
+Machine learning is an extremely rich field, in spite of its young +age. The increases we have seen during the last three decades in +computational capabilities have been followed by developments of +methods and techniques for analyzing and handling large date sets, +relying heavily on statistics, computer science and mathematics. The +field is rather new and developing rapidly. Popular software packages +written in Python for machine learning like +Scikit-learn, +Tensorflow, +PyTorch and Keras, all +freely available at their respective GitHub sites, encompass +communities of developers in the thousands or more. And the number of +code developers and contributors keeps increasing. Not all the algorithms and methods can be given a rigorous mathematical -justification, opening up thereby large rooms for experimenting -and trial and error and thereby exciting new developments. -However, a solid command of linear algebra, multivariate theory, -probability theory, statistical data analysis, -understanding errors and Monte Carlo methods are central elements in a proper understanding of many of -algorithms and methods we will discuss. +justification, opening up thereby large rooms for experimenting and +trial and error and thereby exciting new developments. However, a +solid command of linear algebra, multivariate theory, probability +theory, statistical data analysis, understanding errors and Monte +Carlo methods are central elements in a proper understanding of many +of algorithms and methods we will discuss.
@@ -139,7 +146,7 @@ algorithms and methods we will discuss.
-These lectures aim at giving you an overview of central aspects of +These setsof lectures aim at giving you an overview of central aspects of statistical data analysis as well as some of the central algorithms used in machine learning. We will introduce a variety of central algorithms and methods essential for studies of data analysis and @@ -190,23 +197,22 @@ and jackknife methods. The second part of the material covers several algorithms used in machine learning. -
-
-
-The approaches to machine learning are many, but are often split into two main categories. -In supervised learning we know the answer to a problem, -and let the computer deduce the logic behind it. On the other hand, unsupervised learning -is a method for finding patterns and relationship in data sets without any prior knowledge of the system. -Some authours also operate with a third category, namely reinforcement learning. This is a paradigm -of learning inspired by behavioral psychology, where learning is achieved by trial-and-error, +The approaches to machine learning are many, but are often split into +two main categories. In supervised learning we know the answer to a +problem, and let the computer deduce the logic behind it. On the other +hand, unsupervised learning is a method for finding patterns and +relationship in data sets without any prior knowledge of the system. +Some authours also operate with a third category, namely +reinforcement learning. This is a paradigm of learning inspired by +behavioral psychology, where learning is achieved by trial-and-error, solely from rewards and punishment.
-Another way to categorize machine learning tasks is to consider the desired output of a system. -Some of the most common tasks are: +Another way to categorize machine learning tasks is to consider the +desired output of a system. Some of the most common tasks are:
+The last ingredient is a so-called cost +function which allows us to present an estimate on how good our model +is in reproducing the data it is supposed to train.
Here we will build our machine learning approach on elements of the @@ -234,28 +244,161 @@ machine learning algorithms
-
-
-
-
-
-
-
-
-
+Many of the examples we discuss in this series of lectures come with +existing data files or provide code examples which produce the data to +be analyzed. Most of the applications we will discuss deal with +small data sets (less than a terabyte of information) and can easily +be analyzed and tested on standard off the shelf laptops you find in general +grocery stores. + +
+In most of the cases we will study, we will either generate the data +to analyze ourselves (both for supervised learning and unsupervised +learning) or we will recur again and again to data present in say +scikit-learn or tensorflow. Many of the examples we end up +dealing with are from a privacy and data protection point of view, +rather inoccuous and boring results of numerical +calculations. However, this does not hinder us from developing a sound +ethical attitude to the data we use, how we analyze the data and how +we handle the data. + +
+The most immediate and simplest possible ethical aspects deal with our +approach to the scientific process. Nowadays, with version control +software like Git and various online +repositories like Github, +Gitlab etc, we can easily make our codes +and data sets we have used, freely and easily accessible to a wider +community. This helps us almost automagically in making our science +reproducible. The large open-source development communities involved +in say Scikit-learn, +Tensorflow, +PyTorch and Keras, are +all excellent examples of this. The codes can be tested and improved +upon continuosly, helping thereby our scientific community at large in +developing data analysis and machine learning tools. It is much +easier today to gain traction and acceptance for making your science +reproducible. From a societal stand, this is an important element +since many of the developers are employees of large public institutions like +universities and research labs. Our taxpayer do deserve to get +something back for their bucks. + +
+However, this more mechanical aspect of the ethics of science (in +particular the reproducibility of scientific results) is something +which is obvious and everybody should do as part of the dialectics of +science. The fact that many scientists are not willing to share their codes or +data is detrimental to the scientific discourse. + +
+Before we proceed, we should add a disclaimer. Even though +we may dream of computers developing some kind of higher learning +capabilities, at the end (even if the artificial intelligence +community keeps touting our ears full of fancy futuristic avenues), it is we +who end up constructing and instructing, via various algorithms, the +computers. Self-driving cars for example, rely on sofisticated +programs which take into account all possible situations a car can +encounter. In addition, extensive usage of training datas from GPS +information, maps etc, are typically fed into the software for +self-driving cars. Adding to this various sensors and cameras that +feed information to the programs, there are zillions of ethical issues +which arise from this. + +
+For self-driving cars, where basically many of the standard machine +learning algorithms discussed here enter into the codes, at a certain +stage we have to make choices. Yes, we , the lads and lasses who wrote +a program for a specific brand of a self-driving car. As an example, +a most carmakers have as their utmost priority the security of the +driver and the accompanying passengers. A famous carmaker, which is +one of the leaders in the market of self-driving cars, had if +statements of the following type: suppose there are two obstacles in +front of you and you cannot avoid to collide with one of them. One of +the obstacles is a monstertruck while the other one is a kindergarten +class trying to cross the road. The self-driving car algo would then +opt for the hitting the small folks instead of the monstertruck, since +the likelihood of surving a collision with our future citizens, is +much higher. + +
+This brings us leads then to serious ethical aspects. Why should we +opt for such an option? Who decides and who is entitled to make such +choices? Keep in mind that many of the algorithms you will about in +this series of lectures or hear about later, are indeed based on +simple programming instructions. And you are very likely to be one of +the people who may end up writing such a code. Thus, developing a +sound ethical attitude to what we do, an approach well beyond the +simple mechanistic one of making our science available and +reproducible, is much needed. The example of the self-driving cars is +just one of infinitely many cases where we have to make choices. When +you analyze data on economic inequalities, who guarantees that you are +not weighting some data in a particular way, perhaps because you dearly want a +specific conclusion which may support your political views? + +
+We do not have the answers here, but we want you think over these +topics in a more overarching way. A statistical data analysis with +its dry numbers and graphs meant to guide the eye, do not necessarily +reflect the truth, whatever that is. As a scientist, and after a +university education, you are supposedly a better citizen, with an +improved critical view and understanding of the scientific method, and +perhaps some deeper understandings of the ethics of science at +large. Use these insights. Be a critical citizen. You owe it to our +societies. + +
+To do: Add references and acknowledgements diff --git a/doc/pub/Introduction/html/Introduction.html b/doc/pub/Introduction/html/Introduction.html index 971aa236b..478112798 100644 --- a/doc/pub/Introduction/html/Introduction.html +++ b/doc/pub/Introduction/html/Introduction.html @@ -6,9 +6,9 @@ Automatically generated HTML file from DocOnce source
- + -