Natural Language Processing and Machine Learning Basics

Department of CSE, NIT Trichy
An Introduction to
Natural Language Processing
and
Machine Learning
Karthik Sankar
Department of CSE
NIT Trichy
November 11, 2009 Artificial Intelligence 1

Natural Language Processing

Artificial Intelligence
A lot of human communication is by means of natural language
So computers could be a ton more useful if they could read our email, do our
library research, chat to us, do all of these things involve dealing with natural
language
They're pretty good at dealing with machine languages that are made for them,
but human languages, not so.
“Look. The computer just can't deal with the kind of stuff that humans produce,
and how they naturally interact”
We’re exploiting human cleverness rather than working out how to have
computer cleverness.
November 11, 2009 Department of CSE, NIT Trichy 3

Definition
NLP is a field of computer science and linguistics concerned with the interactions
between computers and human (natural) languages
Categories
 Phonology - study of speech sounds

 Morphology - study of meaningful components of words
 Syntax - study of structural relationships between words
 Semantics - study of meaning

Phonology
Modeling the pronunciation of a word as a string of symbols – PHONES
Articulatory Phonetics: How phones are produced as the various organs in the
mouth, throat and nose modify the airflow from the lungs.
Can
Chair
Coach
Syllables

Morphology
Identification, analysis and description of the structure of words.
Inflections
Number dog/dogs : goose/geese
Tense hunt – hunted
Case his - hers
Gender
Person
Word Formation
mother in law
hot dog
Finite State Machines

Finite State Transducers

Syntax
Part of Speech Tagging
Noun
Verb
Adjective …
I can write – aux. verb OR verb OR noun
Context Free Grammars

Semantics
Understanding and representing the meaning
having
Who has What does he

have
First Order Predicate Calculus
Has(Ram, book)

Ambiguity
Adjective: the adjectives are associated with which of the two nouns ?
“pretty little girls' school”
Pronoun: which noun does ‘they’ relate to ?

We gave the monkeys the bananas because they were hungry.
We gave the monkeys the bananas because they were over-ripe.
Emphasis: notice the change in meaning due to the change in stress

I never said she stole my money

Ambiguity - contd
Fed raises interest rates half a percent in effort to control inflation
She rates highly

Our water rates are high
Japanese movies interest me

The interest rate is 8 percent
Fed raises
The raises we received was small

Resolving Ambiguity
 Part of Speech Tagging
 Word Sense Disambiguation
 Probabilistic Parsing
 Speech Act Interpretation

Perceptions
Perception provides agents with information about the world they inhabit.
Perception is initiated by sensors.
A sensor is anything that can record some aspect of the environment and pass it
as input to an agent program.
The sensor could be as simple as a one-bit sensor that detects whether a switch
is on or off or as complex as the retina of the human eye, which contains more
than a hundred million photosensitive elements
 Image processing
 Computer Vision
 Speech recognition
 Facial recognition
 Object recognition

Applications
 Information retrieval & Web Search
Information retrieval (IR) is the science of searching for documents,
for information within documents, and for metadata about documents, as well
as that of searching databases and the World Wide Web.
 Information Extraction
Information extraction (IE) is a type of information retrieval whose goal is to
automatically extract structured information, i.e. categorized and contextually
and semantically well-defined data from a certain domain, from
unstructured machine-readable documents
 Question Answering
Type in keywords to Asking Questions in Natural Language.
Response from documents to extracted or generated answer
 Text Summarization
Process of distilling most important information from a source to produce an
abridged version
 Machine Translation
use of computer software to translate text or speech from one natural
language to another.
Applications
 Speech - recognition & synthesis
Deriving a textual representation of a spoken utterance
 Natural Language understanding and generation
NLG system is like a translator that converts a computer based representation
into a natural language representation.
 Human - Computer Conversation
Dialogue between humans and computers using natural language.
 Text Generation
A method for generating sentences from “keywords” or “headwords”.
 Hand writing recognition
Ability of a computer to receive and interpret intelligible handwritten input
from sources such as paper documents, photographs, touch-screens and other
devices

Machine Learning

Machine Learning
The ability to learn
 Learning something new

 Learning something new about something you already knew
 Learning how to do something better, either more efficiently or with more
accuracy
A system can improve its problem solving accuracy (and possibly efficiency) by
learning how to do something better

Types of Machine Learning - 1
Symbolic
Explicitly represented Domain knowledge
Sub-Symbolic or Connectionist Networks

• Neural Networks
• simulate the structure and/or functional aspects of
biological neural networks
• Simple processing elements (neurons), which can
exhibit complex global behaviour, determined by
the connections between the processing elements
and element parameters
Genetic and Evolutionary Learning

Learning through adaptation

Types of Machine Learning - 2 - “is there a teacher ???”
Supervised
Training data is available
Unsupervised
Training data is not available. Self learning process
Reinforcement
how an agent ought to take actions in an environment so as to maximize some
notion of long-term reward

Types of Machine Learning - 3
Knowledge acquisition
Learning through problem solving
Explanation based learning
Analogy

Framework for Symbol Based Learning
 Data and the goals of the learning task

 The representation of Learned Language
 A set of operations
 The concept space
 Heuristic Search

Framework for Symbol Based Learning

Example - the goal is to build an arch
positive positive
negative
negative

Example

Example

Example

Version Space Search

Concept space

Generalization Operations
Color(ball, red)
generalizes to Color(X, red)
Shape(X, round) ^ Size(X, small) ^ Color(X, red)

generalizes to Shape(X, round) ^ Size(X, small)
Covering
p covers q

Candidate Elimination Algorithm
 Specific to general direction

 General to specific direction
 Bi-directional

Role of negative examples

Version Space Search - Specific to general direction

Version Space Search - Specific to general direction - example

Version Space Search - General to specific direction

Version Space Search - General to specific direction - example



How the algorithm works

Thank you

Natural Language Processing and Machine Learning Basics

Uploaded by

Natural Language Processing and Machine Learning Basics

Uploaded by

Department of CSE, NIT Trichy

November 11, 2009 Artificial Intelligence 1

Natural Language Processing

November 11, 2009 Artificial Intelligence 2

A lot of human communication is by means of natural language

November 11, 2009 Department of CSE, NIT Trichy 3

 Phonology - study of speech sounds

November 11, 2009 Department of CSE, NIT Trichy 4

November 11, 2009 Department of CSE, NIT Trichy 5

Finite State Machines

November 11, 2009 Department of CSE, NIT Trichy 6

I can write – aux. verb OR verb OR noun

Context Free Grammars

November 11, 2009 Department of CSE, NIT Trichy 7

Who has What does he

First Order Predicate Calculus

November 11, 2009 Department of CSE, NIT Trichy 8

Pronoun: which noun does ‘they’ relate to ?

Emphasis: notice the change in meaning due to the change in stress

November 11, 2009 Department of CSE, NIT Trichy 9

Fed raises interest rates half a percent in effort to control inflation

She rates highly

Japanese movies interest me

November 11, 2009 Department of CSE, NIT Trichy 10

 Part of Speech Tagging

 Word Sense Disambiguation

 Speech Act Interpretation

November 11, 2009 Department of CSE, NIT Trichy 11

Perception is initiated by sensors.

November 11, 2009 Department of CSE, NIT Trichy 12

November 11, 2009 Department of CSE, NIT Trichy 14

November 11, 2009 Artificial Intelligence 15

 Learning something new

November 11, 2009 Department of CSE, NIT Trichy 16

Types of Machine Learning - 1

Sub-Symbolic or Connectionist Networks

Genetic and Evolutionary Learning

November 11, 2009 Department of CSE, NIT Trichy 17

Types of Machine Learning - 2 - “is there a teacher ???”

November 11, 2009 Department of CSE, NIT Trichy 18

Types of Machine Learning - 3

Learning through problem solving

Explanation based learning

November 11, 2009 Department of CSE, NIT Trichy 19

Framework for Symbol Based Learning

 Data and the goals of the learning task

November 11, 2009 Department of CSE, NIT Trichy 20

Framework for Symbol Based Learning

November 11, 2009 Department of CSE, NIT Trichy 21

Example - the goal is to build an arch

November 11, 2009 Department of CSE, NIT Trichy 22

November 11, 2009 Department of CSE, NIT Trichy 23

November 11, 2009 Department of CSE, NIT Trichy 24

November 11, 2009 Department of CSE, NIT Trichy 25

Version Space Search

November 11, 2009 Department of CSE, NIT Trichy 26

Version Space Search

Shape(X, round) ^ Size(X, small) ^ Color(X, red)

November 11, 2009 Department of CSE, NIT Trichy 27

Version Space Search

Candidate Elimination Algorithm

 Specific to general direction

November 11, 2009 Department of CSE, NIT Trichy 28

Version Space Search

Role of negative examples

November 11, 2009 Department of CSE, NIT Trichy 29

Version Space Search - Specific to general direction

November 11, 2009 Department of CSE, NIT Trichy 30

Version Space Search - Specific to general direction - example

November 11, 2009 Department of CSE, NIT Trichy 31

Version Space Search - General to specific direction

November 11, 2009 Department of CSE, NIT Trichy 32

Version Space Search - General to specific direction - example

November 11, 2009 Department of CSE, NIT Trichy 33

November 11, 2009 Department of CSE, NIT Trichy 34

Version Space Search