Free lesson · Time series

Markov chains: a two-state model you can calculate by hand

Open interactive lesson Practice calculations Explore labs

Start with the idea

A Markov chain describes movement between a small set of states. Imagine a machine that is either running normally or under strain. Each state has a list of probabilities for the next step.

Symbols, units & horizon

q_t: row vector of current state probabilities
q_(t,i): probability of state i today
A_ij: probability of tomorrow’s state j given today’s state i
i,j: source and destination state indices
Σ_i: sum over all possible source states
t: one fixed daily step
all values: dimensionless probabilities

When and why to use this

Use a small state model to understand regime persistence, scenario transitions and the environment behind sequential decision methods.

A transition is a move from one state to another, including staying in the same state. Draw two circles, calm and stress, and label the arrows. If a calm day stays calm with probability .9, its probability of moving to stress must be .1 in this two-state model.

A transition matrix stores those arrows: each row names today’s state, each column names tomorrow’s state, and each row sums to one. A probability vector describes uncertainty about today’s state. To predict tomorrow’s state, add all routes that lead to it.

The Markov assumption says the chosen current state contains the historical information needed for the next-state distribution. Real markets need not satisfy this assumption. If duration or older observations still matter, expand the state or consider a richer model rather than calling the data Markov by definition.

This lesson assumes you can observe or specify the current state. In the next lesson you will infer an unobserved state from noisy measurements. Estimated transition probabilities must come from past data; retrospective labels and transitions can leak future information.

q_{t + 1, j} = \sum_{i} q_{t, i} A_{i j}, q_{t + 1} = q_{t} A

Model assumptions, derivation and arithmetic

Markov chains: a two-state model you can calculate by hand

For each possible source state, multiply its current probability by the conditional transition to the desired destination.
Add these mutually exclusive routes. This is the law of total probability.
Repeat for every destination. Writing the same weighted sums together gives row-vector multiplication q_t A. Verify the results are nonnegative and sum to one.

Work it by hand

Let q=[.8,.2] for calm/stress and A=[[.9,.1],[.3,.7]]. Tomorrow’s calm probability is .8×.9+.2×.3=.78. Stress probability is .8×.1+.2×.7=.22. They sum to one.

Apply it in a strategy

State what each state means and whether it is observed or inferred.
Estimate and inspect transition counts, including rare transitions and changing conditions.
Compare next-step predictions with a constant-frequency baseline on later observations.

Research deliverable

Draw a two-state transition diagram and calculate one-step probabilities by enumerating routes.

Sources & evidence · reviewed 12 September 2026

The next lesson links and documents the supplied textbook appendix. Its Markov-chain progression informs this introduction; the market examples and hand calculations are original teaching cases.

Further reading: Jurafsky & Martin · Hidden Markov Models, Appendix A ↗

Python implementation

Self-contained teaching example. Python 3.10+; dependencies and input conventions are shown in the code and notation. Run in your own Python environment.

from math import isfinite

def markov_predict(probabilities,transition):
    n=len(probabilities)
    if not n or len(transition)!=n or any(len(row)!=n for row in transition): raise ValueError("Square transition required")
    for row in [probabilities]+list(transition):
        if not all(isfinite(x) and 0<=x<=1 for x in row) or abs(sum(row)-1)>1e-9: raise ValueError("Probability rows must sum to one")
    return [sum(probabilities[i]*transition[i][j] for i in range(n)) for j in range(n)]

print(markov_predict([.8,.2],[[.9,.1],[.3,.7]]))

Continue learning

Time Series Analysis — all lessons

Quantitative finance and development glossary · Python resources and libraries · Research sources and limitations