mhjensen
197 lines
9.1 KiB
Plaintext
197 lines
9.1 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"<!-- dom:TITLE: Data Analysis and Machine Learning: Introduction and Representing data -->\n",
|
|
"# Data Analysis and Machine Learning: Introduction and Representing data\n",
|
|
"<!-- dom:AUTHOR: Morten Hjorth-Jensen at Department of Physics, University of Oslo & Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University -->\n",
|
|
"<!-- Author: --> \n",
|
|
"**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n",
|
|
"\n",
|
|
"Date: **Nov 27, 2017**\n",
|
|
"\n",
|
|
"Copyright 1999-2017, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"## What is Machine Learning?\n",
|
|
"\n",
|
|
"Machine learning is the science of giving computers the ability to\n",
|
|
"learn without being explicitly programmed. The idea is that there\n",
|
|
"exist generic algorithms which can be used to find patterns in a broad\n",
|
|
"class of data sets without having to write code specifically for each\n",
|
|
"problem. The algorithm will build its own logic based on the data.\n",
|
|
"\n",
|
|
"Machine learning is a subfield of computer science, and is closely\n",
|
|
"related to computational statistics. It evolved from the study of\n",
|
|
"pattern recognition in artificial intelligence (AI) research, and has\n",
|
|
"made contributions to AI tasks like computer vision, natural language\n",
|
|
"processing and speech recognition. It has also, especially in later\n",
|
|
"years, found applications in a wide variety of other areas, including\n",
|
|
"bioinformatics, economy, physics, finance and marketing.\n",
|
|
"\n",
|
|
"## Types of Machine Learning\n",
|
|
"\n",
|
|
"\n",
|
|
"The approaches to machine learning are many, but are often split into two main categories. \n",
|
|
"In *supervised learning* we know the answer to a problem,\n",
|
|
"and let the computer deduce the logic behind it. On the other hand, *unsupervised learning*\n",
|
|
"is a method for finding patterns and relationship in data sets without any prior knowledge of the system.\n",
|
|
"Some authours also operate with a third category, namely *reinforcement learning*. This is a paradigm \n",
|
|
"of learning inspired by behavioural psychology, where learning is achieved by trial-and-error, \n",
|
|
"solely from rewards and punishment.\n",
|
|
"\n",
|
|
"Another way to categorize machine learning tasks is to consider the desired output of a system.\n",
|
|
"Some of the most common tasks are:\n",
|
|
"\n",
|
|
" * Classification: Outputs are divided into two or more classes. The goal is to produce a model that assigns inputs into one of these classes. An example is to identify digits based on pictures of hand-written ones. Classification is typically supervised learning.\n",
|
|
"\n",
|
|
" * Regression: Finding a functional relationship between an input data set and a reference data set. The goal is to construct a function that maps input data to continuous output values.\n",
|
|
"\n",
|
|
" * Clustering: Data are divided into groups with certain common traits, without knowing the different groups beforehand. It is thus a form of unsupervised learning.\n",
|
|
"\n",
|
|
"## Different algorithms\n",
|
|
"In this course we will build our machine learning approach on a statistical foundation, with elements \n",
|
|
"from data analysis, stochastic processes etc before we proceed with the following machine learning algorithms\n",
|
|
"\n",
|
|
"1. Linear regression and its variants\n",
|
|
"\n",
|
|
"2. Decision tree algorithms, from simpler to more complex ones\n",
|
|
"\n",
|
|
"3. Nearest neighbors models\n",
|
|
"\n",
|
|
"4. Bayesian statistics \n",
|
|
"\n",
|
|
"5. Support vector machines and finally various variants of\n",
|
|
"\n",
|
|
"6. Artifical neural networks\n",
|
|
"\n",
|
|
"Before we proceed however, there are several practicalities with data analysis and software tools we would \n",
|
|
"like to present. These tools will help us in our understanding of various machine learning algorithms. \n",
|
|
"\n",
|
|
"Our emphasis here is on understanding the mathematical aspects of different algorithms, however, where possible \n",
|
|
"we will emphasize the importance of using available software. \n",
|
|
"\n",
|
|
"\n",
|
|
"## Software and needed installations\n",
|
|
"We will make intensive use of python as programming language and the myriad of available libraries. \n",
|
|
"Furthermore, you will find IPython/Jupyter notebooks invaluable in your work. \n",
|
|
"You can run **R** codes in the Jupyter/IPython notebooks, with the immediate benefit of visualizing your data.\n",
|
|
"\n",
|
|
"\n",
|
|
"If you have Python installed (we recommend Python3) and you feel pretty familiar with installing different packages, \n",
|
|
"we recommend that you install the following Python packages via **pip** as\n",
|
|
"1. pip install numpy scipy matplotlib ipython scikit-learn mglearn sympy pandas pillow\n",
|
|
"\n",
|
|
"For Python3, replace **pip** with **pip3**.\n",
|
|
"\n",
|
|
"For OSX user we recommend also, after having installed Xcode, to install **brew**. Brew allows \n",
|
|
"for a seamless installation of additional software via for example\n",
|
|
"1. brew install python3\n",
|
|
"\n",
|
|
"For Linux users, with its variety of distributions like for example the widely popular Ubuntu distribution\n",
|
|
"you can use **pip** as well and simply install Python as \n",
|
|
"1. sudo apt-get install python3 (or python for pyhton2.7)\n",
|
|
"\n",
|
|
"etc etc. \n",
|
|
"\n",
|
|
"## Python installers\n",
|
|
"If you don't want to perform these operations separately, we recommend two widely used distrubutions which set up \n",
|
|
"all relevant dependencies for Python, namely\n",
|
|
"1. anaconda\n",
|
|
"\n",
|
|
"2. Enthought canopy\n",
|
|
"\n",
|
|
"## Installing R and C++\n",
|
|
"\n",
|
|
"You will also find it convenient to utilize R. Say more about R. \n",
|
|
"Jupyter/Ipython notebook allows you run **R** code interactively in your browser. The software library **R** is \n",
|
|
"tuned to statistically analysis and allows for an easy usage of the tools we will discuss in these texts.\n",
|
|
"\n",
|
|
"For the C++ affecianodas, Jupyter/IPython notebook allows you also to install C++ and run codes written in this language \n",
|
|
"interactively in the browser. Since we will emphasize writing many of the algorithms yourself, you can thus opt for\n",
|
|
"either Python or C++ as programming languages. \n",
|
|
"\n",
|
|
"To add more entropy, **cython** can also be used when running your notebooks. It means that Python with the Jupyter/IPython notebook \n",
|
|
"setup allows you to integrate widely popular softwares and tools for scientific computing. With its versatility, \n",
|
|
"including symbolic operations, Python offers a unique computational environment. Your Jupyter/IPython notebook \n",
|
|
"can easily be converted into a nicely rendered **PDF** file or a Latex file for further processing.\n",
|
|
"\n",
|
|
"\n",
|
|
"## Introduction to Jupyter notebook and available tools\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"## Representing data, overarching aims"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"metadata": {
|
|
"collapsed": false
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%matplotlib inline\n",
|
|
"\n",
|
|
"import numpy as np\n",
|
|
"import matplotlib.pyplot as plt\n",
|
|
"from scipy import sparse\n",
|
|
"import pandas as pd\n",
|
|
"from IPython.display import display\n",
|
|
"eye = np.eye(4)\n",
|
|
"print(eye)\n",
|
|
"sparse_mtx = sparse.csr_matrix(eye)\n",
|
|
"print(sparse_mtx)\n",
|
|
"x = np.linspace(-10,10,100)\n",
|
|
"y = np.sin(x)\n",
|
|
"plt.plot(x,y,marker='x')\n",
|
|
"plt.show()\n",
|
|
"data = {'Name': [\"John\", \"Anna\", \"Peter\", \"Linda\"], 'Location': [\"Roma\", \"Napoli\", \"Torino\", \"Milano\"], 'Age':[51, 21, 34, 45]}\n",
|
|
"data_pandas = pd.DataFrame(data)\n",
|
|
"display(data_pandas)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Representing data, more examples"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {
|
|
"collapsed": false
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"import numpy as np\n",
|
|
"import matplotlib.pyplot as plt\n",
|
|
"from scipy import sparse\n",
|
|
"import pandas as pd\n",
|
|
"from IPython.display import display\n",
|
|
"import mglearn\n",
|
|
"import sklearn\n",
|
|
"from sklearn.linear_model import LinearRegression\n",
|
|
"from sklearn.tree import DecisionTreeRegressor\n",
|
|
"x, y = mglearn.datasets.make_wave(n_samples=100)\n",
|
|
"line = np.linspace(-3,3,1000,endpoint=False).reshape(-1,1)\n",
|
|
"reg = DecisionTreeRegressor(min_samples_split=3).fit(x,y)\n",
|
|
"plt.plot(line, reg.predict(line), label=\"decision tree\")\n",
|
|
"regline = LinearRegression().fit(x,y)\n",
|
|
"plt.plot(line, regline.predict(line), label= \"Linear Rgression\")\n",
|
|
"plt.show()"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
}
|