Environment types
• Fully observable (vs. partially observable)
• Deterministic (vs. stochastic)
• Episodic (vs. sequential)
• Static (vs. dynamic)
• Discrete (vs. continuous)
• Single agent (vs. multiagent):
Artificial Intelligence a modern approach 1
Fully observable (vs. partially observable)
• Is everything an agent requires to choose its
actions available to it via its sensors? Perfect or
Full information.
• If so, the environment is fully accessible
• If not, parts of the environment are
inaccessible
• Agent must make informed guesses about world.
• In decision theory: perfect information vs.
imperfect information.
Cross Word Poker Backgammon Taxi driver Part picking robot Image analysis
Fully Partially Partially Partially Fully Fully
Artificial Intelligence a modern approach 2
Deterministic (vs. stochastic)
• Does the change in world state
• Depend only on current state and agent’s action?
• Non-deterministic environments
• Have aspects beyond the control of the agent
• Utility functions have to guess at changes in world
Cross Word Poker Backgammon Taxi driver Part
Part picking robot Image analysis
Deterministic Stochastic Stochastic Stochastic Stochastic Deterministic
Artificial Intelligence a modern approach 3
Episodic (vs. sequential):
• Is the choice of current action
• Dependent on previous actions?
• If not, then the environment is episodic
• In non-episodic environments:
• Agent has to plan ahead:
• Current choice will affect future actions
Cross Word Poker Backgammon Taxi driver Part picking robot Image analysis
Sequential Sequential Sequential Sequential Episodic Episodic
Artificial Intelligence a modern approach 4
Static (vs. dynamic):
• Static environments don’t change
• While the agent is deliberating over what to do
• Dynamic environments do change
• So agent should/could consult the world when choosing
actions
• Alternatively: anticipate the change during deliberation OR
make decision very fast
• Semi-dynamic: If the environment itself does not
change with the passage of time but the agent's
performance score does.
Cross Word Poker Backgammon Taxi driver Part picking robot Image analysis
Static Static Static Dynamic Semi
Dynamic
Another example: off-line route planning vs. on-board navigation system
Artificial Intelligence a modern approach 5
Discrete (vs. continuous)
• A limited number of distinct, clearly defined
percepts and actions vs. a range of values
(continuous)
Cross Word Poker Backgammon Taxi driver Part picking robot Image analysis
Discrete Discrete Discrete Conti Conti
Conti
Artificial Intelligence a modern approach 6
Single agent (vs. multiagent):
• An agent operating by itself in an environment or
there are many agents working together
Cross Word Poker Backgammon Taxi driver Part picking robot Image analysis
Single Multi Multi Single Single
Multi
Artificial Intelligence a modern approach 7
Summary.
Observable Deterministic Episodic Static Discrete Agents
Cross Word Fully Deterministic Sequential Static Discrete Single
Poker Fully Stochastic Sequential Static Discrete Multi
Backgammon Partially Stochastic Sequential Static Discrete Multi
Taxi driver Partially Multi
Stochastic Sequential Dynamic Conti
Part picking robot Partially Stochastic Episodic Dynamic Conti Single
Image analysis Fully Deterministic Episodic Semi Conti Single
Artificial Intelligence a modern approach
