Free lesson · Trading algorithms

Kalman filtering: combine a prediction with a noisy observation

Open interactive lesson Practice calculations Explore labs

Start with the idea

A Kalman filter updates an uncertain hidden state using noisy measurements. It weighs the model prediction against the observation according to their uncertainties.

Symbols, units & horizon

x̂⁻: predicted latent state
z: noisy measurement
P: previous state uncertainty variance
Q: process-noise variance
R: observation-noise variance
P⁻,P⁺: predicted and updated uncertainty
K: scalar Kalman gain
All variances: square of the state/measurement units in this identity-observation example

When and why to use this

Use Kalman filtering when a transparent evolving-state model fits the problem better than a fixed smoother, and when its noise assumptions can be checked.

The simplest scalar model assumes a state follows a random walk and the observed value equals state plus independent noise. Predict by carrying forward the previous state and adding process uncertainty. Update by moving toward the new observation according to the Kalman gain.

Trading applications include latent fair-value estimation, dynamic hedge ratios and noisy factor states. A multivariate model uses matrices and may have regression observations; extended or unscented variants address nonlinear models under additional approximations. A flexible dynamic hedge can absorb divergence rather than discover an edge.

Filtering uses information up to the current observation. Smoothing uses later observations to revise past states and must not be substituted into a historical trading signal. Process and observation noise parameters control responsiveness and should be calibrated only on permitted history.

P^{-} = P + Q, K = \frac{P^{-}}{P^{-} + R}, \hat{x} = {\hat{x}}^{-} + K (z - {\hat{x}}^{-}), P^{+} = (1 - K) P^{-}

Model assumptions, derivation and arithmetic

Kalman filtering: combine a prediction with a noisy observation

Under the random-walk model, retain the predicted state and add independent process variance Q.
Combine independent Gaussian prediction and measurement precisions. Rearranging the precision-weighted mean yields gain K=P⁻/(P⁻+R).
Multiply the measurement surprise z−x̂⁻ by K and add it to the prediction. The posterior variance reduces to (1−K)P⁻ in this scalar model.

Work it by hand

Prediction 100, prior variance 1, process variance 1 and measurement variance 2 give P⁻=2 and K=.5. Observation 104 updates the state to 102 with posterior variance 1.

Apply it in a strategy

Write the state and observation equations, including what changes and what is measured.
Estimate noise scales on past data and compare with EWMA at a similar responsiveness.
Use filtered states in replay; reserve smoothed states for retrospective analysis, clearly labelled.

Research deliverable

Show prediction, observation surprise, gain and state update through a small event sequence, with a filtered-versus-smoothed timing distinction.

Continue from a continuous state estimate to discrete hidden states →

Python implementation

Self-contained teaching example. Python 3.10+; dependencies and input conventions are shown in the code and notation. Run in your own Python environment.

def scalar_kalman(prediction,prior_variance,process_variance,measurement,measurement_variance):
    if min(prior_variance,process_variance)<0 or measurement_variance<=0: raise ValueError("Valid variances required")
    predicted=prior_variance+process_variance
    gain=predicted/(predicted+measurement_variance)
    return prediction+gain*(measurement-prediction),(1-gain)*predicted,gain

print(scalar_kalman(100,1,1,104,2))

Continue learning

Trading Algorithms: A Practical Selection Guide — all lessons

Quantitative finance and development glossary · Python resources and libraries · Research sources and limitations