38  Other Time Series Models

The AR(1) model introduces the key idea of temporal dependence through a simple feedback mechanism. While it captures many important behaviours—such as persistence, shock propagation, and mean reversion—real-world time series often exhibit more complex dynamics.

A variety of models extend this basic idea by introducing different forms of dependence. However, these are outside the scope of this course, so we will only briefly mention them here. You may use these as a starting point for further exploration.

Moving Average Models (MA)

In an AR(1) model, dependence arises through past values of the process. An alternative approach is to build dependence through past shocks.

The MA(\(q\)) model refers to the moving average model of order \(q\), where the current value depends on the current shock and the previous \(q\) shocks, given by:

\[ X_t = \mu + \varepsilon_t + \theta_1 \varepsilon_{t-1} + \cdots + \theta_q \varepsilon_{t-q} = \mu + \varepsilon_t + \sum_{i=1}^q \theta_i \varepsilon_{t-i}, \]

where

  • \(\mu\) is the mean of the process,
  • \(\varepsilon_t\) is the current shock (white noise),
  • \(\theta_i\) are the parameters that determine how past shocks affect the current value.

Here:

  • the current value depends on the current shock and the previous shock,
  • dependence comes from noise, rather than past values,
  • shocks affect the system only for a limited number of periods.

ARMA Models

The ARMA (Autoregressive Moving Average) model combines both ideas:

  • AR component: dependence on past values
  • MA component: dependence on past shocks

This allows the model to capture a wider range of behaviours using a relatively simple structure.

The ARMA(\(p\), \(q\)) model refers to the model with \(p\) autoregressive terms and \(q\) moving-average terms, given by:

\[ X_t = \varepsilon_t + \sum_{i=1}^p \phi_i X_{t-i} + \sum_{j=1}^q \theta_j \varepsilon_{t-j}, \]

where

  • \(\varepsilon_t\) is the current shock (white noise),
  • \(\phi_i\) are the parameters for the AR component,
  • \(\theta_j\) are the parameters for the MA component.

ARIMA Models

Some time series exhibit trends or long-term changes over time. In such cases, the data may not fluctuate around a fixed mean.

ARIMA (Autoregressive Integrated Moving Average) models address this by applying differencing, which transforms the data into a form that behaves more like a stationary time series.

These models are widely used in practice for forecasting.

Nonlinear and Regime-Switching Models

In some systems, the behaviour of the time series may change depending on its current state.

For example:

  • economic dynamics may differ during recessions and expansions,
  • environmental systems may behave differently under extreme conditions.

Models such as threshold autoregressive (TVAR) allow the relationship between past and present to change across different regimes.

Other nonlinear models, such as generalised autoregressive conditional heteroskedasticity (GARCH), capture changing volatility over time, which is common in financial data.

Big Picture

All of these models share a common goal:

to describe how the present depends on the past.

They differ only in how this dependence is introduced:

  • through past values (AR),
  • through past shocks (MA),
  • through combinations of both (ARMA),
  • through transformations of the data (ARIMA),
  • through interactions across variables (VAR),
  • or through nonlinear relationships (TVAR, GARCH).