Chapter 7

Natural Language Processing

Compiled By: Bal Krishna Nyaupane
balkrishnanyaupane@[Link]
.
 Introduction to NLP
 Natural Language Processing (NLP) is both a modern computational technology and a method
of investigating and evaluating claims about human language itself.
 NLP is the branch of computer science focused on developing systems that allow computers
to communicate with people using everyday language.
 NLP is the use of computers to process written and spoken language for some practical useful
purpose.
 NLP refers to AI method of communicating with an intelligent systems using a natural
language such as English.
 Processing of Natural Language is required when you want an intelligent system like robot to
perform as per your instructions, when you want to hear decision from a dialogue based
clinical expert system, etc.
 The field of NLP involves making computers to perform useful tasks with the natural
languages humans use. The input and output of an NLP system can be −Speech and Written
Text.
Dreams ??
 NLP Terminology
 Phonology : It is study of organizing sound systematically.
 Morpheme: It is primitive unit of meaning in a language. e.g. “carry”, “pre”, “ed”, “ly”, “s”.
 Morphology: It is a study of construction of words from primitive meaningful units. Example
• carried  carry + ed (past tense)
• independently  in + (depend + ent) + ly
• Googlers  (Google + er) + s (plural)
 Syntax: It refers to arranging words to make a sentence. It also involves determining the structural role
of words in the sentence and in phrases.
 Semantics: It is concerned with the meaning of words and how to combine words into meaningful
phrases and sentences.
 Pragmatics: It deals with using and understanding sentences in different situations and how the
interpretation of the sentence is affected.
 Discourse: It deals with how the immediately preceding sentence can affect the interpretation of the next
sentence. For example, interpreting pronouns and interpreting the temporal aspects of the information.
 World Knowledge: It includes the general knowledge about the world. What each language user must
know about the other’s beliefs and goals.
 Syntax, Semantic, Pragmatics
 Syntax concerns the proper ordering of words and its affect on meaning.
• The dog bit the boy.
• The boy bit the dog.
• * Bit boy dog the the.
• Colorless green ideas sleep furiously.
 Semantics concerns the (literal) meaning of words, phrases, and sentences.
• “plant” as a photosynthetic organism
• “plant” as a manufacturing facility
• “plant” as the act of sowing
 Pragmatics concerns the overall communicative and social context and its effect
on interpretation.
• The ham sandwich wants another beer. (co-reference, anaphora)
• John thinks vanilla. (ellipsis)
5
 Component to NLP
 There are two components of NLP
• Natural Language Understanding (NLU):Understanding involves the following tasks
Mapping the given input in natural language into useful representations.
Analyzing different aspects of the language: morphological analysis, syntactic
analysis, semantic analysis, discourse analysis, …
• Natural Language Generation (NLG):It is the process of producing meaningful phrases
and sentences in the form of natural language from some internal representation.
• NLG involves:
Text planning − It includes retrieving the relevant content from knowledge base.
Sentence planning − It includes choosing required words, forming meaningful
phrases, setting tone of the sentence.
Text Realization − It is mapping sentence plan into sentence structure.
• Different level of synthesis required: deep planning (what to say), syntactic generation
 Why NL Understanding is hard?
 NL Understanding is much harder than NL Generation. But, still both of them are hard.
 Natural language is extremely rich in form and structure.
 It is very ambiguous. There can be different levels of ambiguity −
• Lexical ambiguity: It is at very primitive level such as word-level. For example, treating
the word “board” as noun or verb?
• Syntax Level ambiguity: A sentence can be parsed in different ways. For example, “He
lifted the beetle with red cap.” − Did he use cap to lift the beetle or he lifted a beetle that
had red cap?
• Referential ambiguity: Referring to something using pronouns. For example, Rima went
to Gauri. She said, “I am tired.” − Exactly who is tired?
 Many input can mean the same thing.
 Interaction among components of the input is not clear.
 Steps in NLP
 Lexical Analysis
• It involves identifying and analyzing the structure of words.
• Lexicon of a language means the collection of words and
phrases in a language.
• Lexical analysis is dividing the whole chunk of text into
paragraphs, sentences, and words.
 Syntactic Analysis (Parsing)
 It involves analysis of words in the sentence for grammar and
arranging words in a manner that shows the relationship among
the words.
 Syntactic analysis takes an input sentence and produces a
representation of its grammatical structure.
 A grammar describes the valid parts of speech of a language and
how to combine them into phrases. The grammar of English is
nearly context free.
 The sentence such as “The school goes to boy” is rejected by
English syntactic analyzer.
 Steps in NLP
 Implementation Aspects of Syntactic Analysis
• Context-Free Grammar: It is the grammar that consists rules with a single symbol on the
left-hand side of the rewrite rules.
• A parse tree breaks down the sentence into structured parts so that the computer can easily
understand and process it. Each level of the tree corresponds to the application of one
grammar rule. In order for the parsing algorithm to construct this parse tree, a set of rewrite
rules, which describe what tree structures are legal, need to be constructed.
• Grammar rule:
S → NP VP
Articles (DET) → a | an | the
Nouns (N) → bird | birds | grain | grains
Noun Phrase (NP) → Article + Noun | Article + Adjective + Noun
Verbs (V) → pecks | pecking | pecked
Verb Phrase (VP) → NP V | V NP
Adjectives (ADJ) → beautiful | small | chirping
 Steps in NLP
 Semantic Analysis
• It draws the exact meaning or the dictionary meaning from the text.
• It is done by mapping syntactic structures and objects in the task domain.
• Semantic analysis is a process of converting the syntactic representations into a meaning
representation.
• This involves the following tasks:
 Word sense determination
 Sentence level analysis
• Word sense: Words have different meanings in different contexts.
• Example: Mary had a bat in her office.
 bat = `a baseball thing’ or bat = `a flying mammal’
• Sentence Level Meaning: Once the words are understood, the sentence must be assigned some
meaning. Examples
 She saw her duck.
 I saw a man with a telescope.
• Non-examples: Colorless green ideas sleep furiously. This would be rejected semantically as
colorless green would make no sense.
 Steps in NLP
 Discourse Integration
• The meaning of any sentence depends upon the meaning of the sentence just before it. In
addition, it also brings about the meaning of immediately succeeding sentence.
 Pragmatic Analysis
• During this, what was said is re-interpreted on what it actually meant. It involves deriving
those aspects of language which require real world knowledge.
• Pragmatics comprises aspects of meaning that depend upon the context or upon facts
about real world. These aspects include:
 Pronouns and referring expressions
 Logical inferences that can be drawn from the meanings of a set of propositions.
 Discourse structure: the meaning of a collection of sentences taken together.
• Example
• Handling Pronouns: “Mary eats apples. She likes them.”
She=“Mary”, them=“apples”.
• Handling ambiguity: Pragmatic ambiguity: “you’re late”.
What’s the speaker’s intention: informing or criticizing?
Example: Pragmatic Analysis
 Phonotic Analysis
 Speech processing
• Humans process speech remarkably well.
• Speech interface can replace keyboards and monitors.
• Convert Acoustic signals to Text.
• Phonemes are the smallest recognizable speech unit in a language.
• Graphemes are the textual representation.
• Phonemes can be identified using their phonetic & spectral features.
• Pronunciation of different speakers
• Pace of speech
• Speech ambiguity – Homonyms
 I ate eight cakes.
 That band is banned.
 I went to the mall near by to buy some food.
 The Finnish were the first ones to finish
 I know no James Bond.
 NLP Applications
 Information Retrieval/Detection: To search and retrieve documents in response to queries for
information.
 Passage Retrieval: To search and retrieve part of documents in response to queries for information.
 Information Extraction: To extract information that fits pre-defined database schemas or templates,
specifying the output formats.
 Question/Answering Tasks: To answer general questions by using texts as knowledge base: Fact
retrieval, combination of IR and IE.
 Text Understanding: To understand texts as people do: Artificial Intelligence.
 Text-to-Speech & Speech recognition
 Natural Language Dialogue Interfaces to Databases
 Document Classification
 Document Image Analysis
 Automatic Summarization
 Text Proofreading – Spelling & Grammar
 Machine Translation
 NLP Problems
 The same expression means different things in different context.
• Where’s the water? ( Chemistry lab? Must be pure)
• Where’s the water? ( Thirsty? Must be drinking water)
• Where’s the water? ( Leaky roof? It can be dirty)
 No natural language program can be complete because of new words, expression, and meaning
can be generated quite freely.
• I’ll fax it to you
 There are lots of ways to say the same thing.
• Ram was born on October 11.
• Ram’s birthday is October 11.
 Sentence and phrases might have hidden meanings
• “Out of sight, out of mind”-> “ invisible idiot”
• “The spirit was willing but the flesh was weak” - > “ the vodka was good, but the meat was
bad”
 NLP Problems
 Problem due to syntax and semantics
 Problem due to extensive use of pronouns. (semantic issue)
• E.g. Ravi went to the supermarket. He found his favorite brand of coffee in
rack. He paid for it and left.
• It denotes??
 Use of grammatically incorrect sentence
• He rice eats. (syntax issue)
 Use of conjunctions to avoid repetition of phrases cause problem in NLP
• E.g. Ram and Hari went to restaurant. While Ram had a cup of coffee, Hari had
tea. Hari had a cup of tea.
 Machine Vision
 Machine vision (MV) is the technology and methods used to provide imaging-based automatic
inspection and analysis for such applications as automatic inspection, process control, and robot
guidance in industry.
 Deals with the development of the theoretical and algorithmic basis by which useful information
about the 3D world can be automatically extracted and analyzed from a single or multiple o 2D
images of the world.
 The goal of Machine Vision is to create a model of the real world from images
• A machine vision system recovers useful information about a scene from its two
dimensional projections
• The world is three dimensional
• Two dimensional digitized image
Machine Vision Stages
 Image Processing :Various procedures
such as image enhancement (filtering,
edge detection, surface detection,
computation of depth) Image restoration
(remove point/pattern degradation: there
exist a mathematical expression of the
type of degradation like e.g. Added
multiplicative noise, sin/cos pattern
degradation etc.) takes place.
 Machine Vision Stages
 Image Segmentation: Classify pixels into
groups (regions/objects of interest) sharing
common characteristics.
• Intensity/Color, texture, motion etc.

 Pattern Recognition: Classify an image (region) into one of

a number of known classes
• Statistical pattern recognition (the measurements form
vectors which are classified into classes);
• Structural pattern recognition (decompose the image into
 Image Analysis: Take useful primitive structures).
measurements from pixels, regions, spatial
relationships, motion etc.
• Grey scale / color intensity values;
• Size, distance;
• Velocity;
 Machine Vision Applications
 Medical Applications
• Assist a physician to reach a diagnosis.
• Construct 2D, 3D anatomy models of the human body.
• CG geometric models.
• Analyze the image to extract useful features.
 Remote Sensing
• Take images from high altitudes (from aircrafts, satellites).
• Find ships in the aerial image of the dock.
• Find if new ships have arrived.
• What kind of ships?
 Robotics
 Meteorology
 Quality inspection
Machine Vision Applications