Introduction to
Machine
Learning
1
What is Machine Learning?
“Learning is any process by which a system
improves performance from experience.”
- Herbert Simon
Definition by Tom Mitchell (1998):
Machine Learning is the study of algorithms that
• improve their performance P
• at some task T
• with experience E.
A well-defined learning task is given by <P, T,
E>.
3
Traditional
Programming
Data Output
Computer
Program
Machine
Learning
Data Program
Computer
Output
4
When Do We Use Machine
Learning?
ML is used when:
• Human expertise does not exist (navigating on Mars)
• Humans can’t explain their expertise (speech recognition)
• Models must be customized (personalized medicine)
• Models are based on huge amounts of data (genomics)
Learning isn’t always useful:
• There is no need to “learn” to calculate payroll
5
A classic example of a task that requires machine
learning: It is very hard to say what makes
a2
6
Some more examples of tasks that are
best solved by using a learning
algorithm
• Recognizing patterns:
– Facial identities or facial expressions
– Handwritten or spoken words
– Medical images
• Generating patterns:
– Generating images or motion sequences
• Recognizing anomalies:
– Unusual credit card transactions
– Unusual patterns of sensor readings in a nuclear power
plant
• Prediction:
7
– Future stock prices or currency exchange rates
8
Sample Applications
• Web search
• Computational biology
• Finance
• E-commerce
• Space exploration
• Robotics
• Information extraction
• Social networks
• Debugging software
• [Your favorite area]
8
Slide credit: Pedro
Samuel’s Checkers-Player
“Machine Learning: Field of study that gives
computers the ability to learn without being
explicitly programmed.” -Arthur Samuel
(1959)
9
Defining the Learning Task
Improve on task T, with respect to
performance metric P, based on
experience E
T: Playing checkers
P: Percentage of games won against an arbitrary
opponent E: Playing practice games against itself
T: Recognizing hand-written words
P: Percentage of words correctly classified
E: Database of human-labeled images of handwritten words
T: Driving on four-lane highways using vision sensors
P: Average distance traveled before a human-judged error
E: A sequence of images and steering commands
recorded while observing a human driver.
T: Categorize email messages as spam or
legitimate. P: Percentage of email messages
correctly classified. E: Database of emails, some
with human-given labels
10
State of the Art Applications of
Machine Learning
11
Autonomous Cars
• Nevada made it legal for
autonomous cars to drive
on roads in June 2011
• As of 2013, four states
(Nevada, Florida, California,
and Michigan) have legalized
autonomous cars
Penn’s Autonomous Car
12
(Ben Franklin Racing Team)
Autonomous Car Sensors
13
Autonomous Car Technology
Path
Plannin
g
Laser Terrain
Mapping
Learning from Human Adaptive
Drivers Vision
Sebastian
Stanley
Images and movies taken from Sebastian Thrun’s multimedia w1e4bsite.
Deep Learning in the Headlines
1
5
Deep Belief Net on Face Images
object models
object parts
(combination
of edges)
edges
Andrew Ng
Based on
materials by
pixels
16
Learning of Object Parts
17
Slide credit: Andrew
Training on Multiple Objects
Trained on 4 classes (cars,
faces, motorbikes, airplanes).
Second layer: Shared-
features and object-specific
features.
Third layer: More specific
features.
18
Slide credit: Andrew
Scene Labeling via Deep
Learning
[Farabet et al. ICML 2012, PAMI 2013] 19
Inference from Deep Learned
Models
Generating posterior samples from faces by “filling in”experiments
(cf. Lee and Mumford, 2003). Combine bottom-up and top-down inference.
Input images
Samples from
feedforward
Inference
(control)
Samples
from Full
posterior
inference
20
Machine Learning in
Automatic Speech Recognition
A Typical Speech Recognition System
ML used to predict of phone states from the sound spectrogram
Deep learning has state-of-the-art results
# Hidden Layers 1 2 4 8 10 12
Word Error Rate % 16. 12. 11. 10. 11. 11.
0 8 4 9 0 1
Baseline GMM performance = 15.4%
[Zeiler et al. “On rectified linear units for speech
recognition” ICASSP 2013]
21
Impact of Deep Learning in Speech
Technology
22
Types of Learning
23
Types of Learning
• Supervised (inductive) learning
– Given: training data + desired outputs (labels)
• Unsupervised learning
– Given: training data (without desired outputs)
• Semi-supervised learning
– Given: training data + a few desired outputs
• Reinforcement learning
– Rewards from sequence of actions
24
Supervised Learning:
Regression
• Given (x1, y1), (x2, y2), ..., (xn, yn)
• Learn a function f(x) to predict y given x
– y is real-valued == regression
9
8
Extent (1,000,000 sq
September Arctic Sea Ice
7
6
5
4
km)
3
2
1
0
1970 1980 1990 2000 2010 2020
Year
26
Data from G. Witt. Journal of Statistics Education, Volume 21, Number 1 (2013)
Supervised Learning:
Classification
• Given (x1, y1), (x2, y2), ..., (xn, yn)
• Learn a function f(x) to predict y given x
– y is categorical == classification
Breast Cancer (Malignant / Benign)
1(Malignant)
0(Benign)
Tumor
Size
27
Based on example by Andrew Ng
Supervised Learning:
Classification
• Given (x1, y1), (x2, y2), ..., (xn, yn)
• Learn a function f(x) to predict y given x
– y is categorical == classification
Breast Cancer (Malignant / Benign)
1(Malignant)
0(Benign)
Tumor
Size
Tumor Size 28
Supervised Learning:
Classification
• Given (x1, y1), (x2, y2), ..., (xn, yn)
• Learn a function f(x) to predict y given x
– y is categorical == classification
Breast Cancer (Malignant / Benign)
1(Malignant)
0(Benign)
Tumor
Predict Size
Benign Predict Malignant
Tumor Size 29
Supervised Learning
• x can be multi-dimensional
– Each dimension corresponds to an attribute
- Clump Thickness
- Uniformity of Cell Size
Ag - Uniformity of Cell Shape
e
…
Tumor Size
30
Based on example by Andrew Ng
Unsupervised Learning
• Given x1, x2, ..., xn (without labels)
• Output hidden structure behind the x’s
– E.g., clustering
31
Unsupervised Learning
Genomics application: group individuals by genetic similarity
Gene
s
Individuals 32
[Source: Daphne Koller]
Unsupervised Learning
Organize computing Social network analysis
clusters
Image credit: NASA/JPL-Caltech/E. Churchwell (Univ. of dison
Wisconsin, Ma )
Market Astronomical data analysis 33
Slid segmentation
e credit: Andrew Ng
Unsupervised Learning
• Independent component analysis –
separate a combined signal into its
original sources
34
Image credit: statsoft.com Audio from
Unsupervised Learning
• Independent component analysis –
separate a combined signal into its
original sources
35
Image credit: statsoft.com Audio from
Reinforcement Learning
• Given a sequence of states and actions
with (delayed) rewards, output a policy
– Policy is a mapping from states actions
that tells you what to do in a given state
• Examples:
– Credit assignment problem
– Game playing
– Robot in a maze
– Balance a pole on your hand
36
The Agent-Environment Interface
Agent and environment interact at discrete time steps : t 0, 1, 2, K
Agent observes state at step t : st S
produces action at step t : at A(st )
gets resulting reward : rt1
and resulting next state : st 1
... rt +1 rt +2 rt +3 s ...
st a st at st at t +3a
+1 +2 t
t +1 +2 +3
37
Slide credit: Sutton & Barto
Reinforcement Learning
https://www.youtube.com/watch?v=4cgWya-wjgY
38
Inverse Reinforcement Learning
• Learn policy from user demonstrations
Stanford Autonomous Helicopter
http://heli.stanford.edu/
https://www.youtube.com/watch?v=VCdxqn0fc
39
nE
40
Framing a Learning Problem
41
Designing a Learning System
• Choose the training experience
• Choose exactly what is to be learned
– i.e. the target function
• Choose how to represent the target function
• Choose a learning algorithm to infer the
target function from the experience
Training Learner
data
Environment
/ Experience Knowledg
e
Testing
data Performance
Element 41
Moone
Based on slide by Ray
y
Training vs. Test Distribution
• We generally assume that the training
and test examples are independently
drawn from the same overall distribution
of data
– We call this “i.i.d” which stands for
“independent and identically distributed”
• If examples are not independent, requires
collective classification
• If test distribution is different, requires
transfer learning
42
ML in a Nutshell
• Tens of thousands of machine
learning algorithms
– Hundreds new every year
• Every ML algorithm has three components:
– Representation
– Optimization
– Evaluation
43
Slide credit: Pedro Domingos
Various Function
Representations
• Numerical functions
– Linear regression
– Neural networks
– Support vector machines
• Symbolic functions
– Decision trees
– Rules in propositional logic
– Rules in first-order predicate logic
• Instance-based functions
– Nearest-neighbor
– Case-based
• Probabilistic Graphical Models
– Naïve Bayes
– Bayesian networks
– Hidden-Markov Models (HMMs)
– Probabilistic Context Free Grammars (PCFGs)
44
Slide credit: Ray
– Markov networks
45
Slide credit: Ray
Various Search/Optimization
Algorithms
• Gradient descent
– Perceptron
– Backpropagation
• Dynamic Programming
– HMM Learning
– PCFG Learning
• Divide and Conquer
– Decision tree induction
– Rule learning
• Evolutionary Computation
– Genetic Algorithms (GAs)
– Genetic Programming (GP)
– Neuro-evolution
46
Slide credit: Ray
Evaluation
• Accurac
y
• Precision and recall
• Squared error
• Likelihood
• Posterior probability
• Cost / Utility
• Margin
• Entropy
• K-L divergence
• etc.
4
Slide credit: Pedro 7
ML in Practice
• Understand domain, prior knowledge, and goals
• Data integration, selection, cleaning, pre-processing,
Loop etc.
• Learn models
• Interpret results
• Consolidate and deploy discovered knowledge
4
Based on a slide by Pedro 8
Lessons Learned about Learning
• Learning can be viewed as using direct or
indirect experience to approximate a chosen
target function.
• Function approximation can be viewed as a
search through a space of hypotheses
(representations of functions) for one that best
fits a set of training data.
• Different learning methods assume different
hypothesis spaces (representation languages)
and/or employ different search techniques.
Slide credit: Ray
49
Slide credit: Ray
A Brief History
of Machine
Learning
50
History of Machine Learning
• 1950s
– Samuel’s checker player
– Selfridge’s Pandemonium
• 1960s:
– Neural networks: Perceptron
– Pattern recognition
– Learning in the limit theory
– Minsky and Papert prove limitations of Perceptron
• 1970s:
– Symbolic concept induction
– Winston’s arch learner
– Expert systems and the knowledge acquisition bottleneck
– Quinlan’s ID3
– Michalski’s AQ and soybean diagnosis
– Scientific discovery with BACON
– Mathematical discovery with AM
51
Slide credit: Ray
History of Machine Learning
(cont.)
• 1980s:
– Advanced decision tree and rule learning
– Explanation-based Learning (EBL)
– Learning and planning and problem solving
– Utility problem
– Analogy
– Cognitive architectures
– Resurgence of neural networks (connectionism, backpropagation)
– Valiant’s PAC Learning Theory
– Focus on experimental methodology
• 1990s
– Data mining
– Adaptive software agents and web applications
– Text learning
– Reinforcement learning (RL)
– Inductive Logic Programming (ILP)
52
Slide credit: Ray
– Ensembles: Bagging, Boosting, and Stacking
– Bayes Net learning
53
Slide credit: Ray
History of Machine Learning
(cont.)
• 2000s
– Support vector machines & kernel methods
– Graphical models
– Statistical relational learning
– Transfer learning
– Sequence labeling
– Collective classification and structured outputs
– Computer Systems Applications (Compilers, Debugging, Graphics, Security)
– E-mail management
– Personalized assistants that learn
– Learning in robotics and vision
• 2010s
– Deep learning systems
– Learning for big data
– Bayesian methods
– Multi-task & lifelong learning
53
Based on slide by Ray
– Applications to vision, speech, social networks, learning to read,
etc.
– ???
54
Based on slide by Ray
What We’ll Cover in this
Course
• Supervised learning • Unsupervised learning
– Decision tree induction – Clustering
– Linear regression – Dimensionality
– Logistic regression reduction
– Support vector • Reinforcement
machines & kernel
methods learning
– Model ensembles – Temporal
difference
– Bayesian learning learning
– Neural networks & – Q learning
deep learning
– Learning theory • Evaluation
• Applications
Our focus will be on applying machine learning to real
applications
54
