Kalman Filter State-Space Trend Extraction Engine

Every noisy series hides a state you cannot observe directly: the true level, the true trend. The Kalman filter is the provably optimal way to recover it in a linear Gaussian world, and this engine runs the real recursions, filter, smoother and exact likelihood, on a process you control.

How To Use This Model

Reading This Tool

This tool simulates a local linear trend, a hidden level that drifts with a hidden, slowly-changing slope, buried under measurement noise you control. It then runs the exact Kalman filter forward through the data and the Rauch-Tung-Striebel smoother backward, and scores both against the true hidden path it knows but the filter never sees.

The point to internalize: the filter uses only past data (real-time estimation), the smoother uses the full sample (historical reconstruction), and the smoother always wins. The gap between them is the value of hindsight, made precise.

Process & Noise Parameters

Observations 300

Measurement Noise σ_ε 2.0

Level Noise σ_η 0.30

Slope Noise σ_ζ 0.020

Random Seed 11

The filter is given the true noise variances, so it operates at its theoretical optimum. The signal-to-noise ratio q = σ²_η/σ²_ε is what governs how aggressively the filter chases new observations versus trusting its own forecast.

Estimation Accuracy Against The Hidden Truth

q = –

Raw Observation RMSE

Kalman Filter RMSE

RTS Smoother RMSE

Exact Log-Likelihood

Observations, True Hidden Level, Filtered & Smoothed Estimates

The Slope Nobody Ever Observes

True Terminal Slope

Smoothed Terminal Slope

Slope Tracking RMSE

Noise Reduction vs. Raw Data

True Slope Path vs. Smoothed Slope Estimate

Time Series Econometrics & Signal Extraction

The State-Space Form & Recursions

The local linear trend is the canonical structural time-series model of Harvey (1989): a random-walk level driven by a random-walk slope, observed with error. The Kalman filter delivers the minimum-mean-squared-error estimate of the hidden state given data up to time t; the Rauch-Tung-Striebel backward pass then conditions every estimate on the full sample. The log-likelihood shown is the exact Gaussian likelihood via the prediction-error decomposition, with the first observations treated as a diffuse burn-in. This implementation’s steady-state behavior was verified against the closed-form Riccati solution for the local-level special case.

When To Actually Use This Model

Extracting a defensible trend from noisy operational or economic data, revenue run-rates, sensor streams, macroeconomic indicators, without the arbitrary window choices of moving averages.
Estimating time-varying regression coefficients (a beta that drifts) by putting the coefficient in the state vector.
Teaching maximum likelihood for latent-variable time series: the prediction-error decomposition on this page is the likelihood used across dynamic factor models, DSGE estimation and affine term-structure models.

Key Assumptions & Limitations

Linearity and Gaussianity: with non-linear dynamics or heavy-tailed noise the Kalman filter is no longer optimal, extended, unscented and particle filters exist for those regimes.
Known variances: here the filter is handed the true noise parameters. In practice they are estimated by maximizing the very likelihood this tool computes, which introduces estimation uncertainty the filtered bands do not show.
The smoother is not available in real time by definition, using smoothed estimates in a backtest of a trading rule is look-ahead bias, one of the most common errors in applied work.

Foundational References

Kalman, R. E. (1960). A New Approach to Linear Filtering and Prediction Problems. Journal of Basic Engineering, 82(1), 35-45.

Harvey, A. C. (1989). Forecasting, Structural Time Series Models and the Kalman Filter. Cambridge University Press.

Durbin, J., & Koopman, S. J. (2012). Time Series Analysis by State Space Methods (2nd ed.). Oxford University Press.

Testing for regime shifts instead of extracting trends?

Open The Cointegration & Structural Break Tester →