State Space Models¶
State Space Representation¶
A state space model is a dynamic system of equations
\[\begin{split}\begin{align}
\boldsymbol{\xi}_{t+1} & = F\boldsymbol{\xi}_{t} +
\boldsymbol{v}_{t+1} \\
\boldsymbol{Y}_{t} & = A^{'}\boldsymbol{x}_{t} +
H^{'}\boldsymbol{\xi}_{t} + \boldsymbol{w}_{t} \\
E[\boldsymbol{v}_{t}\boldsymbol{v}_{\tau}^{'}] & =
\bigg\{\begin{array}{c} Q \hspace{10pt} t = \tau \\ \boldsymbol{0}
\hspace{10pt} \text{o/w} \\ \end{array} \\
E[\boldsymbol{w}_{t}\boldsymbol{w}_{\tau}^{'}] & =
\bigg\{\begin{array}{c} R \hspace{10pt} t = \tau \\ \boldsymbol{0}
\hspace{10pt} \text{o/w} \\ \end{array} \\
E[\boldsymbol{v}_{t}\boldsymbol{w}_{\tau}^{'}] & = \boldsymbol{0}
\hspace{4pt} \forall \hspace{4pt} t,\tau.
\end{align}\end{split}\]
State Space Representation¶
- \(\smash{\boldsymbol{Y}_{t}}\) is a vector of \(\smash{n}\) variables observed at \(\smash{t}\).
- \(\smash{\boldsymbol{\xi}_{t}}\) is a vector of \(\smash{r}\) unobserved variables at \(\smash{t}\).
- \(\smash{\boldsymbol{x}_{t}}\) is a vector of exogenous or predetermined variables at \(\smash{t}\).
- The first equation of the system is the state equation.
- The second equation of the system is the observation equation.
- \(\smash{\boldsymbol{v}_{t}}\) and \(\smash{\boldsymbol{w}_{t}}\) are vector WN processes and mutually uncorrelated at all lags.
Example \(\smash{AR(p)}\)¶
The standard form of an \(\smash{AR(p)}\):
\[\smash{Y_{t+1} - \mu = \phi_{1}(Y_{t} - \mu) + \phi_{2}(Y_{t-1} -
\mu) + \ldots + \phi_{p}(Y_{t-p+1} - \mu) + \varepsilon_{t+1}}.\]
Define:
\[\begin{split}\smash{\boldsymbol{\xi}_{t} = \left[\begin{array}{c} Y_{t} - \mu \\
Y_{t-1} - \mu \\ \vdots \\ Y_{t-p+1} - \mu \\ \end{array}
\right], \,\,\,\,\, F = \left[\begin{array}{ccccc} \phi_{1} &
\phi_{2} & \ldots & \phi_{p-1} & \phi_{p} \\ 1 & 0 & \ldots & 0 & 0
\\ \vdots & \vdots & \ddots & \vdots & \vdots \\ 0 & 0 & \ldots & 1
& 0 \\ \end{array} \right], \,\,\,\,\, \boldsymbol{v}_{t} =
\left[\begin{array}{c} \varepsilon_{t} \\ 0 \\ 0 \\ \vdots \\ 0 \\
\end{array} \right],}\end{split}\]
\[\,\,\]
Example \(\smash{AR(p)}\)¶
\[\begin{split}\begin{gather}
\boldsymbol{Y}_{t} = Y_{t}, \hspace{10pt} A^{'} = \mu, \hspace{10pt}
\boldsymbol{x}_{t} = 1, \hspace{10pt} H^{'} = [1 \hspace{3pt} 0
\hspace{3pt} \ldots \hspace{3pt} 0], \hspace{10pt}
\boldsymbol{w}_{t} = 0, \hspace{10pt} R =0, \\
Q = \left[\begin{array}{cccc} \sigma^{2} & 0& \ldots & 0 \\ 0 & 0 &
\ldots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \ldots &
0\\ \end{array} \right].
\end{gather}\end{split}\]
Example \(\smash{ARMA(p,q)}\)¶
The standard form of an \(\smash{ARMA(p,q)}\):
\[\begin{split}\begin{align}
Y_{t+1} - \mu & = \phi_{1}(Y_{t}- \mu) + \ldots +
\phi_{r}(Y_{t-r+1}- \mu) \\
& \hspace{2in} + \varepsilon_{t+1} + \theta_{1}\varepsilon_{t} +
\ldots + \theta_{r-1}\varepsilon_{t-r+2}
\end{align}\end{split}\]
where \(\smash{r = \text{max}\{p,q+1\}}\) and \(\smash{\phi_{j} = 0}\) for \(\smash{j>p}\) and \(\smash{\theta_{j} = 0}\) for \(\smash{j > q}\).
Example \(\smash{ARMA(p,q)}\)¶
Define:
\[\begin{split}\boldsymbol{\xi}_{t} = \left[\begin{array}{c} Y_{t}- \mu \\
\phi_{2}(Y_{t-1}-\mu) + \ldots + \phi_{r}(Y_{t-r+1} - \mu) +
\theta_{1}\varepsilon_{t} + \ldots +
\theta_{r-1}\varepsilon_{t-r+2} \\ \phi_{3}(Y_{t-1}-\mu) + \ldots +
\phi_{r}(Y_{t-r+2} - \mu) + \theta_{2}\varepsilon_{t} + \ldots +
\theta_{r-1}\varepsilon_{t-r+3} \\ \vdots \\ \phi_{r}(Y_{t-1} -
\mu) + \theta_{r-1}\varepsilon_{t} \\ \end{array} \right],\end{split}\]
\[\,\,\]
Example \(\smash{ARMA(p,q)}\)¶
\[\begin{split}\begin{gather}
F = \left[\begin{array}{ccccc} \phi_{1} & 1 & 0 & \ldots & 0
\\ \phi_{2} & 0 & 1 & \ldots & 0 \\ \vdots & \vdots & \ddots &
\ddots & \vdots \\ \phi_{r-1} & 0 & 0 & \ldots & 1 \\ \phi_{r} & 0
& 0 & \ldots & 0 \\ \end{array} \right], \,\,\,\,\,
\boldsymbol{v}_{t} = \left[\begin{array}{c} \varepsilon_{t} \\
\theta_{1}\varepsilon_{t} \\ \vdots \\ \theta_{r-2}\varepsilon_{t} \\
\theta_{r-1}\varepsilon_{t} \\ \end{array}\right], \\
\boldsymbol{Y}_{t} = Y_{t}, \hspace{10pt} A^{'} = \mu, \hspace{10pt}
\boldsymbol{x}_{t} = 1, \hspace{10pt} H^{'} = [1 \hspace{3pt} 0
\hspace{3pt} \ldots \hspace{3pt} 0], \hspace{10pt}
\boldsymbol{w}_{t} = 0, \hspace{10pt} R = 0.
\end{gather}\end{split}\]
Example \(\smash{ARMA(p,q)}\)¶
Alternatively, define \(\smash{\boldsymbol{\xi}_t = (\xi_t, \xi_{t-1}, \ldots, \xi_{t-r+1})^{'}}\) and
\[\begin{split}\begin{gather}
F =
\left[\begin{array}{ccccc} \phi_1 & \phi_2 & \ldots &
\phi_{r-1} & \phi_r \\ 1 & 0 & \ldots & 0 & 0 \\ 0 & 1 &
\ldots & 0 & 0 \\ \vdots & \vdots & \ddots &
\vdots & \vdots \\ 0 & 0 & \ldots & 1 & 0 \end{array}
\right], \,\,\,\,\, \boldsymbol{v}_t = \left[\begin{array}{c}
\varepsilon_t \\ 0 \\ 0 \\ \vdots \\ 0 \end{array} \right], \\
\boldsymbol{Y}_{t} = Y_{t}, \hspace{10pt} A^{'} = \mu, \hspace{10pt}
\boldsymbol{x}_{t} = 1, \hspace{10pt} H^{'} = [1 \hspace{3pt} \theta_1
\hspace{3pt} \ldots \hspace{3pt} \theta_{r-1}], \hspace{10pt}
\boldsymbol{w}_{t} = 0, \hspace{10pt} R =0, \\
Q = \left[\begin{array}{cccc} \sigma^{2} & 0& \ldots & 0 \\ 0 & 0 &
\ldots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \ldots &
0\\ \end{array} \right].
\end{gather}\end{split}\]
Example \(\smash{ARMA(p,q)}\)¶
Then, by the state equation
\[\begin{split}\begin{gather}
\boldsymbol{\xi}_t = F \boldsymbol{\xi}_{t-1} + \boldsymbol{v}_t \\
\implies \phi_r(L) \xi_t = \varepsilon_t,
\end{gather}\end{split}\]
and the observation equation
\[\begin{split}\begin{gather}
\boldsymbol{Y}_t = A^{'} \boldsymbol{x}_t + H^{'}
\boldsymbol{\xi}_t + \boldsymbol{w}_t \\
\implies (Y_t - \mu) = \theta_r(L) \xi_t.
\end{gather}\end{split}\]
Example \(\smash{ARMA(p,q)}\)¶
Combining these two equations,
\[\begin{split}\begin{align}
\phi_r(L) (Y_t-\mu) & = \theta_r(L) \phi_r(L) \xi_t \\
& = \theta_r(L) \varepsilon_t,
\end{align}\end{split}\]
which is the equation for an \(\smash{ARMA(p,q)}\).