Vector Autoregression¶
Definition¶
A \(\smash{p}\) th order vector autoregression generalizes a scalar \(\smash{AR(p)}\):
\[\smash{\boldsymbol{Y}_{t} = \boldsymbol{c} +
\Phi_{1}\boldsymbol{Y}_{t-1} + \Phi_{2}\boldsymbol{Y}_{t-2} +
\ldots + + \Phi_{p}\boldsymbol{Y}_{t-p} +
\boldsymbol{\varepsilon}_{t}}.\]
- \(\smash{Y_{t} = (Y_{1t},Y_{2t},\ldots,Y_{nt})^{'}}\) is an \(\smash{n \times 1}\) vector of random variables.
- \(\smash{\boldsymbol{c} = (c_1,c_2,\ldots,c_n)^{'}}\) is an \(\smash{n \times 1}\) vector of constants.
- \(\smash{\Phi_{j}}\) is an \(\smash{n \times n}\) matrix of autoregressive coefficients for \(\smash{j = 1,\ldots,p}\).
- \(\smash{\boldsymbol{\varepsilon}_{t} = (\varepsilon_{1t},\ldots, \varepsilon_{nt})^{'} }\) is a vector white noise process:
\[\begin{split}\smash{E[\boldsymbol{\varepsilon}_{t}] = \boldsymbol{0} \,\,\, \text{and}
\,\,\,
E[\boldsymbol{\varepsilon}_{t}\boldsymbol{\varepsilon}^{'}_{\tau}] =
\bigg\{\begin{array}{c} \Omega \hspace{10pt} t = \tau \\ 0
\hspace{10pt} \text{o/w} \\ \end{array}}.\end{split}\]
- \(\smash{\boldsymbol{Y}_{t}}\) is referred to as a \(\smash{VAR(p)}\).
\(\smash{AR(p)}\) as \(\smash{VAR(1)}\)¶
Recall, an \(\smash{AR(p)}\) can be written as a \(\smash{VAR(1)}\)
\[\smash{\boldsymbol{Y}_{t} = \Phi \boldsymbol{Y}_{t-1} +
\boldsymbol{v}_{t}}\]
where
\[\begin{split}\smash{\boldsymbol{Y}_{t} = \left[\begin{array}{c} Y_{t} \\ \vdots
\\ Y_{t-p+1} \\ \end{array} \right], \hspace{10pt}
\Phi = \left[\begin{array}{ccccc} \phi_{1} & \phi_{2}
& \ldots & \phi_{p-1} & \phi_{p} \\ 1 & 0 & \ldots & 0 & 0 \\
\vdots & \vdots & \ddots & \vdots & \vdots \\ 0 & 0 & \ldots
& 1 & 0 \\ \end{array} \right], \hspace{10pt}
\boldsymbol{v}_{t} = \left[\begin{array}{c}
\varepsilon_{t} \\ 0 \\ 0 \\ \vdots \\ 0 \\ \end{array}
\right]}\end{split}\]
\[\,\,\]
Lag Operator Notation¶
In lag operator notation,
\[\begin{split}\begin{align*}
\Phi(L) \boldsymbol{Y}_{t} & = \left[I_{n} - \Phi_{1}L -
\Phi_{2}L^{2} - \ldots - \Phi_{p}L^{p}\right]\boldsymbol{Y}_{t}
\\
& = \boldsymbol{c} + \boldsymbol{\varepsilon}_{t}.
\end{align*}\end{split}\]
- \(\smash{\Phi(L)}\) is a matrix where each component is a scalar lag polynomial.
Weak Stationarity¶
The concept of weak stationarity is unchanged: \(\smash{\boldsymbol{Y}_{t}}\) is weakly stationary if
\[\smash{E[\boldsymbol{Y}_{t}] \,\,\, \text{and} \,\,\,
E[\boldsymbol{Y}_{t}\boldsymbol{Y}_{t-j}^{'}]}\]
are independent of \(\smash{t \,\, \forall \,\, j}\)
Mean¶
By weak stationarity,
\[\begin{split}\begin{align*}
E[\boldsymbol{Y}_{t}] & = \boldsymbol{\mu} = \boldsymbol{c} +
\Phi_{1}\boldsymbol{\mu} + \ldots + \Phi_{p}\boldsymbol{\mu} \\
\implies \boldsymbol{\mu} & = [I_{n} - \Phi_{1} - \ldots -
\Phi_{p}]^{-1}\boldsymbol{c}
\end{align*}\end{split}\]
Note that
\[\smash{\boldsymbol{\mu} = (\mu_{1},\mu_{2},\ldots,\mu_{n})^{'} \neq
(\mu,\mu,\ldots,\mu)^{'}}\]
Alternatively, we can re-express as a zero-mean process:
\[\smash{(\boldsymbol{Y}_{t} - \boldsymbol{\mu}) =
\Phi_{1}(\boldsymbol{Y}_{t-1} - \boldsymbol{\mu}) + \ldots +
\Phi_{p}(\boldsymbol{Y}_{t-p} - \boldsymbol{\mu}) +
\boldsymbol{\varepsilon}_{t}}.\]
\(\smash{VAR(p)}\) as \(\smash{VAR(1)}\)¶
We can write a \(\smash{VAR(p)}\) as a \(\smash{VAR(1)}\):
\[\smash{\boldsymbol{\xi}_{t} = F\boldsymbol{\xi}_{t-1} +
\boldsymbol{v}_{t}}\]
where
\[\begin{split}\smash{\boldsymbol{\xi}_{t} = \left[\begin{array}{c}
\boldsymbol{Y}_{t} - \boldsymbol{\mu} \\ \boldsymbol{Y}_{t-1} -
\boldsymbol{\mu} \\ \vdots \\ \boldsymbol{Y}_{t-p+1} -
\boldsymbol{\mu} \\ \end{array} \right], \,\,\,
F = \left[\begin{array}{ccccc} \Phi_{1} & \Phi_{2} &\ldots&
\Phi_{p-1} & \Phi_{p} \\ I_{n} & 0 & 0 & \ldots & 0 \\ 0 &
I_{n} &\ldots & 0 & 0 \\ \vdots & \vdots & \ddots & \vdots & \vdots
\\ 0 & 0 & \ldots & I_{n} & 0 \\
\end{array}\right], \,\,\,
\boldsymbol{v}_{t} = \left[\begin{array}{c}
\boldsymbol{\varepsilon}_{t} \\ 0 \\ 0 \\ \vdots \\ 0 \\
\end{array} \right]}.\end{split}\]
\[\,\,\]
\(\smash{VAR(p)}\) as \(\smash{VAR(1)}\)¶
Clearly,
\[\begin{split}\smash{E[\boldsymbol{v}_{t}\boldsymbol{v}_{\tau}^{'}] =
\bigg\{\begin{array}{c} Q \hspace{10pt} t = \tau \\ 0 \hspace{10pt}
\text{o/w} \\ \end{array}}\end{split}\]
where
\[\begin{split}\smash{Q = \left[\begin{array}{cccc} \Omega & 0& \ldots & 0 \\ 0 &
0 & \ldots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 &
\ldots & 0 \\ \end{array} \right]_{np \times np}}.\end{split}\]
\[\,\,\]
Recursive Iteration¶
Recursively iterating on the \(\smash{VAR(1)}\):
\[\smash{\boldsymbol{\xi}_{t+s} = \boldsymbol{v}_{t+s} +
F\boldsymbol{v}_{t+s-1} + F^{2}\boldsymbol{v}_{t+s-2} + \ldots +
F^{s-1}\boldsymbol{v}_{t+1} + F^{s}\xi_{t}}.\]
Assuming \(\smash{F}\) is nonsingular, it can be decomposed as
\[\smash{F = T \Lambda T^{-1}}.\]
- \(\smash{\Lambda}\) is a diagonal matrix comprised of the \(\smash{np}\) eigenvalues of \(\smash{F}\).
- \(\smash{T}\) is a matrix of eigenvectors as columns.
Recursive Iteration¶
Substituting the decomposition,
\[\begin{split}\begin{gather*}
F^2 = FF = (T \Lambda T^{-1})\,T \Lambda T^{-1} = T \Lambda^{2}
T^{-1} \\
\implies F^{s} = T \Lambda^{s} T^{-1} \rightarrow 0 \,\,\,\,
\text{if} \,\,\,\, |\lambda_{k}| < 1 \,\,\,\, \text{for} \,\,\,\, k
= 1, \ldots, np.
\end{gather*}\end{split}\]
If \(\smash{F^{s}\rightarrow 0}\) as \(\smash{s\rightarrow \infty}\),
- The effect of \(\smash{\boldsymbol{\varepsilon}_{t}}\) on \(\smash{\boldsymbol{\xi}_{t+s}}\) dies out as \(\smash{s\rightarrow \infty}\).
- \(\smash{\boldsymbol{\xi}_{t}}\) (and hence \(\smash{\boldsymbol{Y}_{t}}\)) is stationary and causal.
- Alternatively, \(\smash{\boldsymbol{Y}_{t}}\) is stationary and causal if the roots of \(\smash{[I_{n} - \Phi_{1}z - \Phi_{2}z^{2} - \ldots - \Phi_{p}z^{p}]}\) all lie outside the unit circle.
Vector \(\smash{MA(\infty)}\) Representation¶
If \(\smash{F^{s}\rightarrow 0}\) as \(\smash{s\rightarrow \infty}\), then
\[\smash{\boldsymbol{\xi}_{t+s} = \boldsymbol{v}_{t+s} +
F\boldsymbol{v}_{t+s-1} + F^{2}\boldsymbol{v}_{t+s-2} +
F^{3}\boldsymbol{v}_{t+s-3} + \ldots}\]
which is a vector \(\smash{MA(\infty)}\) process.
Vector \(\smash{MA(\infty)}\) Representation¶
We can also write \(\smash{\boldsymbol{Y}_{t}}\) alone as a vector \(\smash{MA(\infty)}\). First, recognize
\[\begin{split}\begin{align*}
\boldsymbol{Y}_{t+s} & = \boldsymbol{\mu} +
\boldsymbol{\varepsilon}_{t+s} + \Psi_{1}
\boldsymbol{\varepsilon}_{t+s-1} + \Psi_{2}
\boldsymbol{\varepsilon}_{t+s-2} + \ldots + \Psi_{s-1}
\boldsymbol{\varepsilon}_{t+1} \\
& \hspace{0.5in} + F_{11}^{(s)}(\boldsymbol{Y}_{t} -
\boldsymbol{\mu}) + F_{12}^{(s)}(\boldsymbol{Y}_{t-1} -
\boldsymbol{\mu}) + \ldots + F_{1p}^{(s)}(\boldsymbol{Y}_{t-p+1} -
\boldsymbol{\mu}).
\end{align*}\end{split}\]
- \(\smash{\Psi_{j} = F_{11}^{(j)}}\).
- \(\smash{F_{1k}^{(j)}}\) is comprised of rows 1 to \(\smash{n}\) and columns \(\smash{(k-1)n+1}\) to \(\smash{kn}\) of matrix \(\smash{F^{j}}\).
- Note that the matrices \(\smash{(F \times F)[1:n,1:n]}\) and \(\smash{F[1:n,1:n] \times F[1:n,1:n]}\) are not the same.
Vector \(\smash{MA(\infty)}\) Representation¶
Suppose all eigenvalues of \(\smash{F}\) are inside the unit circle.
- Then \(\smash{F^{s}\rightarrow 0}\) as \(\smash{s\rightarrow \infty}\).
- This means \(\smash{F_{1k}^{(s)}\rightarrow 0}\) as \(\smash{s\rightarrow \infty}\).
- In the limit
\[\begin{split}\begin{align*}
\boldsymbol{Y}_{t+s} & = \boldsymbol{\mu} +
\boldsymbol{\varepsilon}_{t+s} + \Psi_{1}
\boldsymbol{\varepsilon}_{t+s-1} + \Psi_{2}
\boldsymbol{\varepsilon}_{t+s-2} + \ldots \\
& = \boldsymbol{\mu} +
\Psi(L)\boldsymbol{\varepsilon}_{t+s}.
\end{align*}\end{split}\]
Inverse of \(\smash{MA(\infty)}\) Lag Polynomial¶
In this case \(\smash{\Psi(L) = \Phi(L)^{-1}}\) or
\[\smash{[1 - \Phi_{1}L - \Phi_{2}L^{2} - \ldots - \Phi_{p}L^{p}][1 +
\Psi_{1}L + \Psi_{2}L^{2} + \ldots] = I_{n}}.\]
Representation with Uncorrelated Noise¶
- In this case the leading matrix \(\smash{J_{0} \neq I_{n}}\).
- The noise vector is uncorrelated:
\[\begin{split}\begin{align*}
E[\boldsymbol{u}_{t}\boldsymbol{u}_{t}^{'}] & =
E[H\boldsymbol{\varepsilon}_{t} \boldsymbol{\varepsilon}_{t}^{'}
H^{'}] \\
& = H E[\boldsymbol{\varepsilon}_{t}\boldsymbol{\varepsilon}_{t}]
H^{'} \\
& = H \Omega H^{'} \\
& = D.
\end{align*}\end{split}\]