In this workshop, we will apply the concepts of AR(1) processes, shock propagation, mean reversion, and ACF analysis through a series of exercises. These exercises will help you understand how to simulate time series data, interpret patterns, and analyse relationships between interacting systems.
Consider a system where current conditions depend on recent past conditions, such as temperature or traffic congestion.
If this were traffic flow, what would \(\phi = 0.6\) suggest about congestion?
A value of \(\phi = 0.6\) suggests moderate persistence in congestion.
In this context:
Interpretation in traffic terms
For example, if traffic is heavily congested now:
Simulate an AR(1) process of length \(n=200\) with \(\phi = 0.6\) and \(\varepsilon_t \sim N(0, 1)\). Plot the simulated time series and its autocorrelation function (ACF). Describe the observed patterns in the time series and ACF.
import numpy as np
import matplotlib.pyplot as plt
from statsmodels.graphics.tsaplots import plot_acf
np.random.seed(1234)
n = 200
phi = 0.6
epsilon = np.random.normal(0, 1, n)
Z = np.zeros(n)
for t in range(1, n):
Z[t] = phi * Z[t-1] + epsilon[t]
fig, ax = plt.subplots(2, 1, figsize=(6, 4))
ax[0].plot(Z)
ax[0].set_title("Simulated AR(1)")
ax[0].set_xlabel("Time")
ax[0].set_ylabel("Z_t")
plot_acf(Z, ax=ax[1], title="ACF of Simulated AR(1)")
plt.tight_layout()
plt.show()
The simulated AR(1) time series exhibits a pattern of persistence, where values tend to be similar to their recent past values due to the positive \(\phi\) coefficient. The ACF shows a gradual decay, indicating that the correlation between values decreases as the lag increases, which is characteristic of an AR(1) process.
import numpy as np
import matplotlib.pyplot as plt
np.random.seed(1234)
phis = [0, 0.5, 0.9, -0.7]
Z_list = {}
for phi in phis:
Z = np.zeros(n)
epsilon = np.random.normal(0, 1, n)
for t in range(1, n):
Z[t] = phi * Z[t-1] + epsilon[t]
Z_list[str(phi)] = Z
fig, ax = plt.subplots(2, 2, figsize=(6, 5))
for i, phi in enumerate(phis):
ax[i//2, i%2].plot(Z_list[str(phi)])
ax[i//2, i%2].set_title(f"AR(1) with phi = {phi}")
ax[i//2, i%2].set_xlabel("Time")
ax[i//2, i%2].set_ylabel("Z_t")
plt.tight_layout()
plt.show()
Suppose a sudden event occurs (e.g., traffic accident or system disruption) that temporarily increases congestion.
In traffic flow context, what does the persistence of the shock represent?
The persistence of the shock represents how long the impact of the disruption continues to affect the system after the event itself has passed.
In the context of traffic:
Simulate an AR(1) process with \(\phi = 0.8\) and \(\varepsilon_t \sim N(0, 1)\) for \(n=100\). Introduce a shock of magnitude 5 at time \(t=50\) (i.e., set \(Z_{50} = Z_{50} + 5\)). Plot the time series and describe how the shock propagates through time. How long does it take for the effect of the shock to diminish? Does the shock disappear immediately or gradually? How does the value of \(\phi\) affect the speed of shock dissipation?
import numpy as np
import matplotlib.pyplot as plt
np.random.seed(123)
n = 100
phi = 0.8
epsilon = np.random.normal(0, 1, n)
Z = np.zeros(n)
for t in range(1, n):
shock = 5 if t == 49 else 0
Z[t] = phi * Z[t-1] + epsilon[t] + shock
plt.plot(Z)
plt.title("AR(1) with Shock at t=50")
plt.xlabel("Time")
plt.ylabel("Z_t")
plt.axvline(x=50, color='red', linestyle='--')
plt.axhline(y=0, color='blue', linestyle='--')
plt.show()
The shock introduced at time \(t=50\) causes a sudden spike in the time series. The effect of the shock does not disappear immediately; instead, it gradually diminishes over time at rate \(\phi^k\) due to the persistence of the AR(1) process. The value of \(\phi = 0.8\) indicates that 80% of the deviation from normal conditions carries forward to the next time step, which means that the shock will have a lasting impact and will take several time steps to dissipate. The higher the value of \(\phi\), the slower the shock will dissipate, as more of its effect is retained in subsequent values.
Many systems tend to return to a normal level after extreme events, such as prices, temperature, or queue length.
What does mean reversion represent in this context?
Mean reversion represents the tendency of the system to return to its typical or equilibrium level after a disturbance.
Simulate an AR(1) process with \(\phi = 0.5\) and \(\varepsilon_t \sim N(0, 1)\) for \(n=100\) with initial value \(Z_1 = 20\). Plot the time series and describe the mean-reverting behaviour. How does the process return to its mean after a deviation?
import numpy as np
import matplotlib.pyplot as plt
np.random.seed(1234)
n = 100
phi = 0.5
epsilon = np.random.normal(0, 1, n)
Z = np.zeros(n)
Z[0] = 20 # initial value
for t in range(1, n):
Z[t] = phi * Z[t-1] + epsilon[t]
plt.plot(Z)
plt.title("AR(1) with Mean Reversion")
plt.xlabel("Time")
plt.ylabel("Z_t")
plt.axhline(y=0, color='red', linestyle='--')
plt.show()
The AR(1) process with \(\phi = 0.5\) exhibits mean-reverting behaviour, where values tend to return towards the mean (which is 0 in this case) after deviations. When the process deviates from the mean, the positive \(\phi\) coefficient causes subsequent values to be influenced by the previous value, leading to a gradual return towards the mean over time. The process will oscillate around the mean, with the magnitude of deviations decreasing as it gets closer to the mean.
import numpy as np
import matplotlib.pyplot as plt
np.random.seed(1234)
n = 100
phi_values = [0.5, 0.9, 1, 1.05]
epsilon = np.random.normal(0, 1, n)
fig, ax = plt.subplots(2, 2, figsize=(6, 4))
for i, phi in enumerate(phi_values):
Z = np.zeros(n)
Z[0] = 20 # initial value
for t in range(1, n):
Z[t] = phi * Z[t-1] + epsilon[t]
ax[i//2, i%2].plot(Z)
ax[i//2, i%2].set_title(f"AR(1) with phi = {phi}")
ax[i//2, i%2].set_xlabel("Time")
ax[i//2, i%2].set_ylabel("Z_t")
ax[i//2, i%2].axhline(y=0, color='red', linestyle='--')
plt.tight_layout()
plt.show() 
The AR(1) process with \(\phi = 0.5\) shows stronger mean reversion compared to the processes with higher \(\phi\) values. In other words, smaller \(|\phi|\) leads to faster mean reversion. As \(\phi\) increases, the speed of mean reversion decreases, and the process becomes more persistent, taking longer to return to the mean after a deviation. When \(\phi = 1\), the process is a random walk and does not revert to the mean at all, while for \(\phi > 1\), the process exhibits explosive behaviour, diverging away from the mean over time.
The ACF is commonly used in practice to detect patterns in time series data, such as traffic flow, temperature, and financial returns.
If you observe slow decay in ACF, what does that suggest about the system?
Observing a slow decay in the ACF suggests that the system has strong persistence (long memory). In the context of:
For \(\phi = 0.5\) and \(\phi = 0.9\), simulate AR(1) processes and plot the time series and ACF. Which ACF decays faster? How does this relate to the value of \(\phi\)? What would you expect the ACF to look like for \(\phi = 0\) and \(\phi = 1\)?
import numpy as np
import matplotlib.pyplot as plt
from statsmodels.graphics.tsaplots import plot_acf
# Reproducibility
np.random.seed(1234)
n = 200
phi_values = [0.5, 0.9]
fig, axes = plt.subplots(2, 2, figsize=(6, 4))
for i, phi in enumerate(phi_values):
epsilon = np.random.normal(0, 1, n)
Z = np.zeros(n)
for t in range(1, n):
Z[t] = phi * Z[t - 1] + epsilon[t]
# Time series plot
axes[i, 0].plot(Z)
axes[i, 0].set_title(f"AR(1) with phi = {phi}")
axes[i, 0].set_xlabel("Time")
axes[i, 0].set_ylabel("Z_t")
# ACF plot
plot_acf(Z, ax=axes[i, 1], lags=40)
axes[i, 1].set_title(f"ACF of AR(1) with phi = {phi}")
plt.tight_layout()
plt.show()
The ACF for \(\phi = 0.5\) decays faster than the ACF for \(\phi = 0.9\), which indicates that the correlation between values decreases more rapidly as the lag increases for \(\phi = 0.5\). This is because a smaller \(\phi\) value leads to less persistence in the time series, while a larger \(\phi\) value results in stronger persistence and a slower decay of the ACF.
Consider two interacting systems, such as:
How does a change in one system affect the other over time? What does the coefficient matrix \(\Phi\) represent in this context?
A change in one system can affect the other through lagged dependence.
In a VAR(1) model,
\[ Z_t = \Phi Z_{t-1} + \varepsilon_t, \]
the current values depend on the past values of both systems.
For example, if the two systems are traffic flow on connected roads:
The coefficient matrix \(\Phi\) represents the dynamic relationship between the systems:
\[ \Phi = \begin{bmatrix} \phi_{11} & \phi_{12} \\ \phi_{21} & \phi_{22} \end{bmatrix} \]
where:
The diagonal entries describe own persistence, while the off-diagonal entries describe cross-system influence.
Consider a bivariate VAR(1) process with coefficient matrix \(\Phi = \begin{bmatrix} 0.5 & 0.2 \\ 0.1 & 0.4 \end{bmatrix}\) and \(\varepsilon_t \sim N(0, I)\). Is the process stable?
For a VAR(1) model to be stable (covariance stationary), the eigenvalues of the coefficient matrix must lie inside the unit circle:
\[ \rho(\Phi) < 1, \]
where \(\rho(\Phi)\) is the spectral radius (largest absolute eigenvalue) of \(\Phi\) calculated as:
\[ \rho(\Phi) = \max_i \{ |\lambda_i| : \lambda \text{ is an eigenvalue of } \Phi \}. \]
The eigenvalues of \(\Phi\) can be calculated as follows:
\[ \begin{aligned} \text{det}(\Phi - \lambda I) &= 0 \\ \text{det}\left(\begin{bmatrix} 0.5 - \lambda & 0.2 \\ 0.1 & 0.4 - \lambda \end{bmatrix}\right) &= 0 \\ (0.5 - \lambda)(0.4 - \lambda) - (0.2)(0.1) &= 0 \\ \lambda^2 - 0.9\lambda + 0.18 &= 0 \\ \lambda_1 = 0.6, \quad \lambda_2 = 0.3 \end{aligned} \]
So:
\[ \rho(\Phi) = \max(|0.6|, |0.3|) = 0.6 < 1, \]
The VAR(1) process is stable since the spectral radius is less than 1.
Simulate a bivariate VAR(1) process with coefficient matrix and \(\varepsilon\) given in part (b). Plot the two time series and their cross-correlation function (CCF). Describe the observed relationships between the two series.
import numpy as np
import matplotlib.pyplot as plt
from statsmodels.graphics.tsaplots import plot_ccf
np.random.seed(1234)
n = 200
Phi = np.array([[0.5, 0.2], [0.1, 0.4]])
epsilon = np.random.normal(0, 1, (2, n))
Z = np.zeros((2, n))
for t in range(1, n):
Z[:, t] = Phi @ Z[:, t-1] + epsilon[:, t]
fig, ax = plt.subplots(2, 1, figsize=(6, 4))
ax[0].plot(Z[0, :], label="Series 1", color="black", linewidth=0.8)
ax[0].plot(Z[1, :], label="Series 2", color="blue", linestyle="--", linewidth=0.8)
ax[0].set_title("Series 1 vs Series 2")
ax[0].set_xlabel("Time")
ax[0].set_ylabel("Z1_t and Z2_t")
ax[0].axhline(0, color='red', linestyle='--')
ax[0].legend(loc="upper right")
plot_ccf(Z[0, :], Z[1, :], ax=ax[1], title="Cross-Correlation Function")
plt.tight_layout()
plt.show()
The two series exhibit some degree of correlation due to the off-diagonal elements in the coefficient matrix \(\Phi\). Series 1 is influenced by both its own past values and the past values of Series 2, while Series 2 is influenced by its own past values and the past values of Series 1. Non-zero cross-correlations indicate that one series influences the other over time.