ARMA Processes¶

\(\smash{ARMA(p,q)}\) Process¶

Given white noise \(\smash{\{\varepsilon_t\}}\), consider the process

\[\smash{Y_t = c + \phi_1 Y_{t-1} + \ldots + \phi_p Y_{t-p} + \varepsilon_t + \theta_1 \varepsilon_{t-1} + \ldots \theta_q \varepsilon_{t-q},}\]

where \(\smash{c}\), \(\smash{\{\phi_i\}_{i=1}^p}\) and \(\smash{\{\theta_i\}_{i=1}^q}\) are constants.

This is an \(\smash{ARMA(p,q)}\) process.

\(\smash{ARMA(p,q)}\) Process¶

We can rewrite in terms of lag operators:

\[\begin{align} \phi(L) Y_t & = c + \theta(L) \varepsilon_t, \end{align}\]

where

\[\begin{split}\begin{align} \phi(L) & = 1-\phi_1 L - \phi_2 L^2 - \ldots - \phi_p L^p \\ \theta(L) & = 1+\theta_1 L + \theta_2 L^2 + \ldots + \theta_q L^q. \end{align}\end{split}\]

\(\smash{ARMA(p,q)}\) as \(\smash{MA(\infty)}\)¶

Recall

\(\smash{\phi(L) = (1-\lambda_1 L)(1-\lambda_2 L) \cdots (1-\lambda_pL).}\)

If the roots, \(\smash{\frac{1}{|\lambda_i|} > 1}\), \(\smash{\forall i}\) then \(\smash{|\lambda_i| < 1}\), \(\smash{\forall i}\) and

\[\begin{split}\begin{align*} \phi(L)^{-1} & = (1-\lambda_1 L)^{-1}(1-\lambda_2 L)^{-1} \cdots (1-\lambda_pL)^{-1} \\ & = \left(\sum_{j=0}^{\infty} \lambda_1^j L^j\right) \left(\sum_{j=0}^{\infty} \lambda_2^j L^j\right) \cdots \left(\sum_{j=0}^{\infty} \lambda_p^j L^j\right). \end{align*}\end{split}\]

\(\smash{ARMA(p,q)}\) as \(\smash{MA(\infty)}\)¶

Thus, if the roots of \(\smash{\phi(L)}\) all lie outside the unit circle,

\[\begin{align*} Y_t & = \mu + \psi(L) \varepsilon_t, \end{align*}\]

where \(\smash{\mu = \phi(L)^{-1} c}\) and \(\smash{\psi(L) = \phi(L)^{-1} \theta(L)}\).

This restriction on the roots of \(\smash{\phi(L)}\) results in

\[\smash{\sum_{i=1}^{\infty} |\psi_i| < \infty.}\]

Hence, \(\smash{Y_t}\) is an \(\smash{MA(\infty)}\) process and is weakly stationary.

The stationarity of an \(\smash{ARMA(p,q)}\) depends only on \(\smash{\{\phi_i\}_{i=1}^p}\) and not on \(\smash{\{\theta_i\}_{i=1}^q}\).

Expectation of \(\smash{ARMA(p,q)}\)¶

Assume \(\smash{Y_t}\) is weakly stationary: the roots of \(\smash{\phi(L)}\) lie outside the unit circle.

\[\begin{split}\begin{align*} \text{E}[Y_t] & = c + \phi_1 \text{E}[Y_{t-1}] + \ldots + \phi_p \text{E}[Y_{t-p}] \\ & \hspace{0.75in} + \text{E}[\varepsilon_t] + \theta_1 \text{E}[\varepsilon_{t-1}] + \ldots + \theta_q \text{E}[\varepsilon_{t-q}]\\ & = c + \phi_1 \text{E}[Y_t] + \ldots + \phi_p \text{E}[Y_t] \\ \Rightarrow \text{E}[Y_t] & = \frac{c}{1-\phi_1 - \ldots - \phi_p} = \mu. \end{align*}\end{split}\]

This is the same mean as an \(\smash{AR(p)}\) process with parameters \(\smash{c}\) and \(\smash{\{\phi_i\}_{i=1}^p}\).

Autocovariances of \(\smash{ARMA(p,q)}\)¶

Given that \(\smash{\mu = c/(1-\phi_1 - \ldots - \phi_p)}\) for weakly stationary \(\smash{Y_t}\):

\[ \begin{align}\begin{aligned}\begin{split}\begin{align*} Y_t & = \mu(1-\phi_1 - \ldots - \phi_p) + \phi_1 Y_{t-1} + \ldots + \phi_p Y_{t-p} \\ & \hspace{2in} + \varepsilon_t + \theta_1 \varepsilon_1 + \ldots \theta_q \varepsilon_{t-q} \\ \Rightarrow & (Y_t - \mu) = \phi_1(Y_{t-1} - \mu) + \ldots + \phi_p(Y_{t-p} - \mu) \\ & \hspace{2in} + \varepsilon_t + \theta_1 \varepsilon_1 + \ldots \theta_q \varepsilon_{t-q}. \end{align*}\end{split}\\\begin{split}\begin{align*} \gamma_j & = \text{E}\left[(Y_t - \mu) (Y_{t-j} - \mu)\right] \\ & = \phi_1 \text{E}\left[(Y_{t-1} - \mu) (Y_{t-j} - \mu)\right] + \ldots \\ & \hspace{0.5in} + \phi_p \text{E}\left[(Y_{t-p} - \mu) (Y_{t-j} - \mu)\right] \\ & \hspace{1in} + \text{E}\left[\varepsilon_t (Y_{t-j} - \mu)\right] + \theta_1 \text{E}\left[\varepsilon_{t-1} (Y_{t-j} - \mu)\right] \\ & \hspace{2.25in} + \ldots + \theta_q \text{E}\left[\varepsilon_{t-q} (Y_{t-j} - \mu)\right] \end{align*}\end{split}\end{aligned}\end{align} \]

Autocovariances of \(\smash{ARMA(p,q)}\)¶

For \(\smash{j > q}\), \(\smash{\gamma_j}\) will follow the same law of motion as for an \(\smash{AR(p)}\) process:

\[\begin{align*} \gamma_j & = \phi_1 \gamma_{j-1} + \ldots + \phi_p \gamma_{j-p} \,\,\,\,\, \text{ for } j = q+1, \ldots \end{align*}\]

These values will not be the same as the \(\smash{AR(p)}\) values for \(\smash{j = q+1, \ldots}\), since the initial \(\smash{\gamma_0, \ldots, \gamma_q}\) will differ.

The first \(\smash{q}\) autocovariances, \(\smash{\gamma_0, \ldots, \gamma_q}\), of an \(\smash{ARMA(p,q)}\) will be more complicated than those of an \(\smash{AR(p)}\).

Redundancy of \(\smash{ARMA(p,q)}\)¶

Factoring the polynomials \(\smash{\phi(L)}\) and \(\smash{\theta(L)}\), an \(\smash{ARMA(p,q)}\) can be written as

\[\begin{align*} (1-\lambda_1 L) \cdots (1-\lambda_p L) (Y_t - \mu) & = (1 - \eta_1 L) \cdots (1 - \eta_q L) \varepsilon_t. \end{align*}\]

If two of the roots are identical, \(\smash{\lambda_i = \eta_j}\), both polynomials can be divided by \(\smash{(1-\lambda_i L)}\).

The result would be an \(\smash{ARMA(p-1, q-1)}\):

\[\begin{align*} (1-\phi_1^* L - \ldots - \phi_{p-1}^* L^{p-1}) (Y_t - \mu) & = (1 + \theta_1^* L + \ldots + \theta_{q-1}^* L^{q-1}) \varepsilon_t. \end{align*}\]