Markov
chain and
Hidden
Markov
Chain
Dr. Bitan Misra
Assistant professor
Techno International New
Town
What is
Markov
chain
• Markov chain refers to mathematical system that
undergoes transitions from one state to other as per
some probabilistic rules.
• The probability of transitioning to any particular state
relies solely on current state and time elapsed.
Transition
matrix
Properties of Markov
Chain
• Irreducibility: Markov chain is irreducible when it is possible to reach any state
from any other state in a finite number of steps.
• Aperiodicity: A Markov chain is aperiodic when it is possible to reach any state
from any other state in a finite number of steps, regardless of the starting state.
• Recurrence: A state in a Markov chain is recurrent if it is possible to return to that
state in a finite number of steps.
• Transience: A state in a Markov chain is transient if it is not possible to return to
that state in a finite number of steps.
• Ergodicity: A Markov chain is ergodic if it is both irreducible and aperiodic and if
the long-term behaviour of the system is independent of the starting state.
• Reversibility: A Markov chain is reversible if probability of transitioning from one
state to another is equal to the probability of transitioning from that state back to
the original state.
Markov chain
Applications of Markov Chains
• One common application of Markov chains is in the field of economics,
where they can be used to model the behaviour of stock prices. In this
case, the possible states of the system might represent upward or
downward trends in the market, and the transitions between states might
be influenced by various economic factors such as interest rates and the
performance of individual stocks. By analyzing the probability of
transitioning between different states, it is possible to make predictions
about the future behaviour of the market.
• Markov chains can be used for modeling biological systems, such as the
spread of a disease through a population. In such case, the states of the
system might represent the number of individuals in a population who are
susceptible, infected, or immune to the disease, and the transitions
between states might be influenced by several factors such as the rate of
infection and the effectiveness of a vaccine. By analyzing the probability of
transitioning between different states, it is possible to make predictions
about the future spread of the disease and the effectiveness of various
control measures.
•
Hidden Markov model
• A Hidden Markov Model (HMM) is a probabilistic model that consists of a
sequence of hidden states, each of which generates an observation. The
hidden states are usually not directly observable, and the goal of HMM is to
estimate the sequence of hidden states based on a sequence of
observations.
Hidden Markov Model
Transition Matrix Emission matrix
Hidden
Markov
Model
Hidden
Markov
Model
Example of HMM
