INTELLIGENT AGENTS

SWE485: Artificial Intelligence Systems

Software Engineering Department

0
*Some slides are by Kris Hauser, IU
OUTLINE
• Definition of agent
• Rational agent
• Properties of environments
• Types of agents

1
 DEFINITION OF AGENT

• Anything that:
• Perceives its environment through sensors
• Acts upon its environment through actuators
• A.k.a. controller, robot

Actions, environment, sensors and actuators

2
DEFINITION OF AGENT

Agent Sensors

Environment
Actuators

3
 DEFINITION OF AGENT

Agent Sensors
Percepts

Environment
This class will deal with the control policy that is
how an agent makes decisions
agent’s behavior is described by the agent
function that maps any given percept
? Those decisions take place many many
times making a loop called
sequence to an action. Control
[percept sequence] action Policy perception action cycle

Actuators
Actions

Sense – Plan - Act

4
EXAMPLES OF AGENTS
• A human agent:
• Sensor: Eyes, ears, and other organs for sensors
VACUUM-CLEANER
• Actuators: Hands, legs, vocal tract, and so on for actuators
• A robotic agent:
• Sensor: Cameras and infrared range finders for sensors
• Actuators: Various motors for actuators
• A software agent:
• Sensor: Receives keystrokes, file contents, and network packets as
sensory inputs
• Actuators: Acts on the environment by displaying on the screen,
writing files, and sending network packets

5
THE VACUUM-CLEANER WORLD

A B

6
 A B
AGENT FUNCTION
[Location, status]

Action causes
change on status

7
AGENT PROGRAM

8
 GOOD BEHAVIOR

COGNITIVE LAWS OF
SCIENCE THOUGHT

TURING RATIONAL
TEST AGENTS
Measure the human do the “right thing” given what it knows 9
success in terms of
GOOD BEHAVIOR
• A rational agent is one that does the right thing, i.e.
rationale

• When an agent is in an environment, it generates a

sequence of actions according to the percepts it receives.

• This sequence of actions causes the environment to go

through a sequence of states.

10
GOOD BEHAVIOR
• The new environment status is desirable, then agent has
performed well.
• Desirable is captured by a performance measure
• There is no one performance measure for all task and agents
• (aka reward, merit, cost, loss, error)
• Part of the problem domain

11
EXAMPLE: PERFORMANCE MEASURE

• Amount of dirt cleaned up in a single eight-

hour shift.
• Reward the agent for having a clean floor
• One point cloud be awarded for each clean
square ( perhaps with a penalty for
electronic consumed and noise generated).

It is better to design performance measures according to

what one actually wants in the environment, rather than
according to how one thinks the agent should behave.
12
RATIONAL AGENT
• What is rational at any given time depends on four things:
• The performance measure that defines the criterion of success.
• The agent’s prior knowledge of the environment.
• The actions that the agent can perform.
• The agent’s percept sequence to date.

• This leads to a definition of a rational agent:

For each possible percept sequence, a rational agent should select an
action that is expected to maximize its performance measure, given the
evidence provided by the percept sequence and whatever built-in
knowledge the agent has.
13
DEFINITION OF ENVIRONMENT
• The real world, or a virtual world
• Example

• Rules of math/formal logic

• Rules of a game
•…
• Specific to the problem domain

14
EXERCISE (DESIGN AGENT)

• In designing an agent, the first step must always be to

specify the task environment (PEAS) as fully as possible

PEAS (Performance, Environment, Actuators, Sensors)

• Formulate the problem domains for:

• A taxi driver
• A doctor diagnosing a patient

15
TASK ENVIRONMENT
PEAS (Performance, Environment, Actuators, Sensors)

16
ENVIRONMENT
• Environments are characterized by:
• Observability (Fully vs Partial)
• Determinism (Deterministic vs. stochastic)
• Episodism (Episodic vs. sequential)
• Stasis (Static vs. dynamic)
• Discreteness (Discrete vs. continuous)
• Number of agents (Single- vs Multi-agent environments)

17
OBSERVABILITY SENSOR

• Fully vs Partial Observable

• If an agent is operating in a fully observable environment, it
can observe the complete state (has a global view) of the
environment at any point in time.
• Examples:
• Tic-Tac-Toe: Fully observable
• Vacuum-Cleaner: Partially observable
• partially observable because of noisy and inaccurate sensors or because parts
of the state are simply missing from the sensor data
• If the agent has no sensors at all then the environment is
unobservable.
18
DETERMINISM
• Deterministic vs. stochastic
• If the next state of the environment is completely determined
by the current state and the action executed by the agent,
then we say the environment is deterministic; otherwise, it is
stochastic.
• Stochastic environment is random in nature and cannot
be determined completely by an agent.
• Example?
In games with dice, you can determine your dice throwing action but not the
outcome of the dice.

• In a deterministic, fully observable environment, agent does

not need to worry about uncertainty.
19
EPISODISM

• Episodic vs. sequential

• Episodic environment, there is a series of one-shot actions, and
only the current percept is required for the action.

• Sequential environment, an agent requires memory of past

actions to determine the next best actions.

• Examples:
• Classification task: Spotting defective parts on assembly line is episodic
• Chess-playing and taxi-driving case are sequential

20
STASIS
• Static vs. dynamic
• If the environment can change while an agent is
deliberating, then we say the environment is dynamic for
that agent; otherwise, it is static
• Static environments are easy to deal because an agent
does not need to continue looking at the world while deciding
for an action.
• Examples:
• Taxi driving is dynamic as the other cars and the taxi itself keep
moving
• Crossword puzzles are static.
21
DISCRETENESS

• Discrete vs. continuous

• An environment is said to be discrete if there is:
• a finite number of actions (left, right, suck up)
• a finite number of percepts (clean, dirty)
• Examples:
• Tic-Tac-Toe and chess game are discrete
• Throwing darts and self-driving car are continuous

22
NUMBER OF AGENTS
• Single vs. Multi-agent environments
• Examples:
• Crossword is a single-agent environment
• Tic-Tac-Toe, chess game is a Multi-agent environment
• Multi-agent environments can be competitive or cooperative.

23
 TYPES OF AGENTS
• Simple reflex (aka reactive, rule-based)
• Model-based
• Goal-based
• Utility-based (aka decision-theoretic, game-theoretic)
• Learning (aka adaptive)

24
SIMPLE REFLEX AGENTS (RULE-BASED)
• Agent selects actions on the basis of the current percept,
ignoring the rest of the percept history.
• Action selected using condition-action rules
• These agents only succeed in the fully observable
environment.
percept
State = percept
(memorized)
state

action

25
RULE-BASED REFLEX AGENT
A B

State = percept

Condition-action rule

if DIRTY = TRUE then SUCK

else if LOCATION = A then RIGHT
26
else if LOCATION = B then LEFT
BUILDING A SIMPLE REFLEX AGENT
• Rules: a map from states to action
• a = π(s)
Learning methods
• Rule can be: Example -> rules

• Designed by hand
• Learned from a “teacher” (e.g., human expert) using ML techniques
• Problems for the simple reflex agent design approach:
• They have very limited intelligence
• Mostly too big to generate and to store.
• Not adaptive to changes in the environment.
• These agents only succeed in the fully observable environment.
27
MODEL-BASED REFLEX AGENTS
• The most effective way to handle partial observability is for
the agent to keep track of the part of the world it can’t see
now.
• A model-based agent has two important factors:
• Model: It is knowledge about "how things happen in the
world," so it is called a Model-based agent.
• Internal State: It is a representation of the current
state based on percept history. State ≠ current percept
• The agent should maintain some sort of internal state that
depends on the percept history and thereby reflects at least
some of the unobserved aspects of the current state
28
 MODEL-BASED REFLEX AGENTS

State estimation

Percepts

Model
Updating the agent state requires information about:
• How the world evolves
• How the agent's action affects the world.
state

actions

1- percept history states

2- Model
29
3- condition-action rules
 MODEL-BASED AGENT
State:
LOCATION Rules:
if LOCATION = A then
HOW-DIRTY(A)
if HAS-SEEN(B) = FALSE then RIGHT
HOW-DIRTY(B)
else if HOW-DIRTY(A) > HOW-DIRTY(B) then SUCK
HAS-SEEN(A) else RIGHT
HAS-SEEN(B) …

Model:
A B
HOW-DIRTY(LOCATION) = X
HAS-SEEN(LOCATION) = TRUE

30
BUILDING A MODEL-BASED REFLEX
AGENT
• A model is a map from prior state s, action a, to new state s’
• s’ = T(s,a)
Can be
• Constructed through domain Percept

knowledge (e.g., rules of a game, Model

state machine of a computer
program, a physics simulator for
a robot)
state
• Learned from watching the
system behave (system
identification, calibration)
• Remember: rules can be action
designed or learned as before 31
MODEL-BASED, GOAL-BASED AGENT
• Knowing something about the current state of the environment is not
always enough to decide what to do.
• For example:
• At a road junction, the taxi can turn left, turn right, or go straight on.
• The correct decision depends on where the taxi is trying to get to.
• So, as well as a current state description, the agent needs some sort of goal
information that describes situations that are desirable.
desirable
• Goal-based agents expand the capabilities of the model-based agent by
having the "goal" information.
• We have model
• We have state
• But do we have condition-action rules? Why?

32
 MODEL-BASED, GOAL-BASED AGENTS

Model

Performance Metric
No condition–
action rules

Planner

action

Search (Chapters 3 to 5) and planning (Chapters 10 and 11) are the subfields
33
of AI devoted to finding action sequences that achieve the agent’s goals.
BUILDING A GOAL-BASED AGENT
• Requires:
• Model of percepts (sensor model)
• Action generation algorithm (planner) Chapter 3!
• Performance metric
• Planning using search
• Performance metric: does it reach the goal? (always)
• 2 Types:
• Problem Solving Agents (use atomic rep. of state)
• Planning Agents (use factored/structured rep. of state)
34
UTILITY-BASED AGENTS
• Goals alone are not enough to generate high-quality behavior
in most environments.
• For example:
• Many action sequences will get the taxi to its destination (thereby
achieving the goal)
• But some are quicker, safer, more reliable, or cheaper than others.
• Goals just provide a crude binary distinction between “happy” and
“unhappy” states.
• A more general performance measure should allow a comparison of
different world states according to exactly how happy they would
make the agent. Because “happy” does not sound very scientific,
economists and computer scientists use the term utility instead.

35
 UTILITY-BASED AGENTS

Model

Performance Metric

Planner

action

No condition– action rules 36

BUILDING A UTILITY-BASED AGENT
• Requires:
• Model of percepts (sensor model)
• Action generation algorithm (planner)
• Performance metric
• Planning using decision theory
• Performance metric: acquire maximum rewards (or
minimum cost)

37
BUILDING A LEARNING AGENT
• Need a mechanism for updating models/rules/planners on-
line as it interacts with the environment
• Need incremental techniques for Machine Learning

38