The autoregressive model of order 1 or AR(1) model is the simplest time series model with temporal dependence, but it captures essential features of many real-world time series, including persistence, shock propagation, and mean reversion. More complex time series models are built by extending this idea, combining multiple lags and interactions. Understanding AR(1) therefore provides the foundation for studying richer dynamic systems.
An AR(1) is a stochastic process indexed by time, defined by the recursive relation
\[ X_t = \phi X_{t-1} + \varepsilon_t, \quad t = 1,2,\dots \]
where:
The AR(1) model introduces dependence through a simple feedback mechanism, where the present is a noisy version of the past.
At each time step, the value of the process is formed by combining:
This simple mechanism transforms white noise into a process with memory and dynamic structure.
The AR(1) model can be interpreted as white noise with feedback. While white noise produces completely independent observations, the AR(1) process feeds past values forward into the future.
This creates temporal dependence:
Unlike white noise, the past now contains information about the future.
Example: Simulate an AR(1) process with (\(\phi = 0.7\)):
\[ X_t = 0.7 X_{t-1} + \varepsilon_t, \quad \varepsilon_t \sim \text{i.i.d. } N(0,1) \]
set.seed(1234)
n <- 1000
phi <- 0.7
epsilon <- rnorm(n, mean = 0, sd = 1)
X <- numeric(n)
X[1] <- 0 # initial value
for (t in 2:n) {
X[t] <- phi * X[t-1] + epsilon[t]
}
plot(X, type = "l",
main = "AR(1) Process (phi = 0.7)",
xlab = "Time", ylab = "Value")
abline(h = 0, col = "red", lty = 2, lwd = 2)
import numpy as np
import matplotlib.pyplot as plt
np.random.seed(1234)
n = 1000
phi = 0.7
epsilon = np.random.normal(0, 1, n)
X = np.zeros(n)
for t in range(1, n):
X[t] = phi * X[t-1] + epsilon[t]
plt.plot(X, color='blue')
plt.axhline(0, color='red', linestyle='--', linewidth=2)
plt.title('AR(1) Process (phi = 0.7)')
plt.xlabel('Time')
plt.ylabel('Value')
plt.show()The resulting plot reveals behaviour very different from white noise:
The parameter \(\phi\) controls how strongly the present value depends on the past. In the AR(1) model,
\[ X_t = \phi X_{t-1} + \varepsilon_t, \]
different choices of \(\phi\) produce very different time series behaviour.
The value of \(\phi\) determines the memory of the process.
Example: Simulate four AR(1) processes using
\[ \phi = 0,\quad 0.5,\quad 0.9,\quad -0.7. \]
Each process is driven by Gaussian white noise:
\[ \varepsilon_t \sim \text{i.i.d. } N(0,1). \]
The following plots illustrate how changing ϕ alters the memory and dynamics of the process.
set.seed(1234)
n <- 300
phis <- c(0, 0.5, 0.9, -0.7)
simulate_ar1 <- function(phi, n) {
epsilon <- rnorm(n, mean = 0, sd = 1)
X <- numeric(n)
X[1] <- 0
for (t in 2:n) {
X[t] <- phi * X[t - 1] + epsilon[t]
}
return(X)
}
par(mfrow = c(2, 2), mar = c(4, 4, 2, 1))
for (phi in phis) {
X <- simulate_ar1(phi, n)
plot(
X,
type = "l",
main = paste("AR(1), phi =", phi),
xlab = "Time",
ylab = "Value"
)
abline(h = 0, col = "red", lty = 2)
}
import numpy as np
import matplotlib.pyplot as plt
np.random.seed(1234)
n = 300
phis = [0, 0.5, 0.9, -0.7]
def simulate_ar1(phi, n):
epsilon = np.random.normal(0, 1, n)
X = np.zeros(n)
for t in range(1, n):
X[t] = phi * X[t - 1] + epsilon[t]
return X
fig, axes = plt.subplots(2, 2, figsize=(10, 6))
for ax, phi in zip(axes.flatten(), phis):
X = simulate_ar1(phi, n)
ax.plot(X)
ax.axhline(0, color="red", linestyle="--")
ax.set_title(f"AR(1), phi = {phi}")
ax.set_xlabel("Time")
ax.set_ylabel("Value")
plt.tight_layout()
plt.show()When \(\phi = 0\), the model becomes \[ X_t = \varepsilon_t. \] The process has no memory. Each value is determined only by the new random shock. This is simply white noise.
When \(\phi = 0.5\), each value carries forward half of the previous value. A positive value tends to be followed by another positive value, but the dependence is not very strong. Shocks persist for a short time before fading away. This produces a time series with mild smoothness and short-term memory.
When \(\phi = 0.9\), most of the previous value is carried forward. The process shows long runs above or below zero. Shocks decay slowly, so the series has strong persistence and appears much smoother than white noise. This is typical of systems where changes are gradual, such as prices, temperatures, demand, or population levels.
When \(\phi = -0.7\), the sign of the previous value tends to reverse. A positive value tends to be followed by a negative value, and a negative value tends to be followed by a positive value. This creates oscillating behaviour around the mean. The process is still stable because \(|\phi| < 1\), but the dependence is negative rather than positive.
In summary, the stable AR(1) model can generate a wide range of behaviours by simply changing the value of \(\phi\):
| Value of \(\phi\) | Behaviour | Interpretation |
|---|---|---|
| \(\phi = 0\) | White noise | No memory |
| \(0 < \phi < 0.5\) | Low to moderate persistence | Short memory, faster decay of shocks |
| \(0.5 \leq \phi < 0.9\) | Moderate to high persistence | Long memory, slower decay of shocks |
| \(\phi \approx 0.9\) or higher | High persistence, close to a unit root | Very long memory, shocks persist for a long time |
| \(\phi < 0\) | Oscillation | Alternating behaviour |
The key idea is that \(\phi\) controls how information from the past is carried into the future. By changing a single parameter, the AR(1) model can generate behaviour ranging from pure randomness to persistent movement and oscillation.
The defining feature of the AR(1) model is its memory. Unlike white noise, where shocks affect only a single time point, in an AR(1) process a shock influences future values through the recursive structure.
To see this, suppose a large positive shock occurs at time \(t\). Then:
\[ X_{t+1} = \phi X_t + \varepsilon_{t+1} \]
so part of that shock carries forward. Substituting recursively, we find:
\[ \begin{aligned} X_{t+2} &= \phi X_{t+1} + \varepsilon_{t+2} \\ &= \phi (\phi X_t + \varepsilon_{t+1}) + \varepsilon_{t+2} \\ &= \phi^2 X_t + \phi \varepsilon_{t+1} + \varepsilon_{t+2} \end{aligned} \]
and continuing this process shows that the impact of the original shock is multiplied by successive powers of \(\phi\).
This reveals an important insight:
This phenomenon is known as shock propagation:
Example: Consider a shock of size 1 at time \(t=1\), and all future shocks are set to zero, plot the propagation of this shock under different values of \(\phi\).
set.seed(1234)
n <- 50
phis <- c(0, 0.5, 0.9, -0.7)
simulate_shock <- function(phi, n) {
X <- numeric(n)
X[1] <- 1 # initial shock
for (t in 2:n) {
X[t] <- phi * X[t-1] # no new noise
}
return(X)
}
par(mfrow = c(2, 2), mar = c(4, 4, 2, 1))
for (phi in phis) {
X <- simulate_shock(phi, n)
plot(
X,
type = "o",
main = paste("Shock propagation, phi =", phi),
xlab = "Time",
ylab = "Value"
)
abline(h = 0, col = "red", lty = 2)
}
import numpy as np
import matplotlib.pyplot as plt
np.random.seed(1234)
n = 50
phis = [0, 0.5, 0.9, -0.7]
def simulate_shock(phi, n):
X = np.zeros(n)
X[0] = 1 # initial shock
for t in range(1, n):
X[t] = phi * X[t-1] # no new noise
return X
fig, axes = plt.subplots(2, 2, figsize=(10, 6))
for ax, phi in zip(axes.flatten(), phis):
X = simulate_shock(phi, n)
ax.plot(X, marker='o')
ax.axhline(0, color="red", linestyle="--")
ax.set_title(f"Shock propagation, phi = {phi}")
ax.set_xlabel("Time")
ax.set_ylabel("Value")
plt.tight_layout()
plt.show()These plots isolate the pure effect of memory:
In all cases with \(|\phi| < 1\), the shock eventually disappears, but the speed and pattern of decay depend critically on \(\phi\).
Shock propagation explains why AR(1) processes exhibit persistence:
A single disturbance does not vanish immediately—it echoes through time, gradually fading according to the system’s memory.
This idea is central to understanding:
A key behaviour exhibited by many AR(1) processes is mean reversion. This refers to the tendency of the process to drift back toward its long-run average over time. In other words, mean reversion describes the long-run outcome of shock propagation: while shocks can cause temporary deviations, the process eventually returns to its mean.
Suppose the process takes on a large positive value at some time (t). Then:
\[ X_{t+1} = \phi X_t + \varepsilon_{t+1} \]
If \(|\phi| < 1\), the term \(\phi X_t\) is smaller in magnitude than \(X_t\). This means:
This stabilising effect is what we call mean reversion.
Mean reversion depends critically on the value of (\(\phi\)):
Thus, the condition \(|\phi| < 1\) ensures that shocks eventually decay and the process returns toward equilibrium.
Example: Visualise mean reversion in an AR(1) process with (\(\phi = 0.7\))
set.seed(1234)
n <- 100
phi <- 0.7
epsilon <- rnorm(n, 0, 1)
X <- numeric(n)
X[1] <- 10 # start far from the mean
for (t in 2:n) {
X[t] <- phi * X[t-1] + epsilon[t]
}
plot(
X,
type = "l",
main = "Mean Reversion in AR(1) (phi = 0.7)",
xlab = "Time",
ylab = "Value"
)
abline(h = 0, col = "red", lty = 2)
import numpy as np
import matplotlib.pyplot as plt
np.random.seed(1234)
n = 100
phi = 0.7
epsilon = np.random.normal(0, 1, n)
X = np.zeros(n)
X[0] = 10 # start far from the mean
for t in range(1, n):
X[t] = phi * X[t-1] + epsilon[t]
plt.plot(X)
plt.axhline(0, color='red', linestyle='--')
plt.title('Mean Reversion in AR(1) (phi = 0.7)')
plt.xlabel('Time')
plt.ylabel('Value')
plt.show()In this model, the long-run mean is zero. From the plot, we observe:
This illustrates that:
deviations from the mean are temporary, not permanent.
| Feature | White Noise | AR(1) |
|---|---|---|
| Dependence | None | Depends on past |
| Memory | No | Yes |
| Shock effect | Immediate only | Persistent |
| Structure | None | Dynamic |
More general autoregressive models allow dependence on multiple past values:
\[ X_t = \phi_1 X_{t-1} + \cdots + \phi_p X_{t-p} + \varepsilon_t = \sum_{j=1}^p \phi_j X_{t-j} + \varepsilon_t, \]
where \(\phi_1, \dots, \phi_p\) are parameters controlling the influence of each lag. The AR(\(p\)) model can capture richer temporal behaviour because the present value may depend on several previous values, not just the most recent one.
Example: Simulate AR(\(p\)) processes with different parameters: \(p = 0, 1, 2, 3\) and various \(\phi\) values.
set.seed(1234)
n <- 300
simulate_ar <- function(phi, n) {
p <- length(phi)
epsilon <- rnorm(n, 0, 1)
X <- numeric(n)
for (t in (p + 1):n) {
X[t] <- sum(phi * X[(t - 1):(t - p)]) + epsilon[t]
}
return(X)
}
models <- list(
"AR(0): White noise" = rnorm(n, 0, 1),
"AR(1): phi = 0.7" = simulate_ar(c(0.7), n),
"AR(2): phi = (0.6, 0.25)" = simulate_ar(c(0.6, 0.25), n),
"AR(3): phi = (0.5, 0.25, -0.2)" = simulate_ar(c(0.5, 0.25, -0.2), n)
)
par(mfrow = c(2, 2), mar = c(4, 4, 2, 1))
for (name in names(models)) {
plot(
models[[name]],
type = "l",
main = name,
xlab = "Time",
ylab = "Value"
)
abline(h = 0, col = "red", lty = 2)
}
import numpy as np
import matplotlib.pyplot as plt
np.random.seed(1234)
n = 300
def simulate_ar(phi, n):
p = len(phi)
epsilon = np.random.normal(0, 1, n)
X = np.zeros(n)
for t in range(p, n):
X[t] = sum(phi[j] * X[t - j - 1] for j in range(p)) + epsilon[t]
return X
models = {
"AR(0): White noise": np.random.normal(0, 1, n),
"AR(1): phi = 0.7": simulate_ar([0.7], n),
"AR(2): phi = (0.6, 0.25)": simulate_ar([0.6, 0.25], n),
"AR(3): phi = (0.5, 0.25, -0.2)": simulate_ar([0.5, 0.25, -0.2], n),
}
fig, axes = plt.subplots(2, 2, figsize=(10, 6))
for ax, (name, series) in zip(axes.flatten(), models.items()):
ax.plot(series)
ax.axhline(0, linestyle="--", color="red")
ax.set_title(name)
ax.set_xlabel("Time")
ax.set_ylabel("Value")
plt.tight_layout()
plt.show()| Model | Example Coefficients | Behaviour |
|---|---|---|
| AR(0) | None | White noise (no memory) |
| AR(1) | \(\phi = 0.7\) | Moderate persistence |
| AR(2) | \(\phi = (0.6, 0.25)\) | Stronger persistence |
| AR(3) | \(\phi = (0.5, 0.25, -0.2)\) | Mixed lag effects |
We will later see that this same pattern appears in the autocorrelation function.↩︎