1. State-Space Model
Transition Equation
x t = A t x t − 1 + W t w t \mathbf{x}_t
=
\mathbf{A}_t\mathbf{x}_{t-1}
+
\mathbf{W}_t\mathbf{w}_t x t = A t x t − 1 + W t w t where:
x t ∈ R n x \mathbf{x}_t\in\mathbb{R}^{n_x} x t ∈ R n x
A t ∈ R n x × n x \mathbf{A}_t\in\mathbb{R}^{n_x\times n_x} A t ∈ R n x × n x x t − 1 \mathbf{x}_{t-1} x t − 1 x t \mathbf{x}_t x t
w t ∈ R n w \mathbf{w}_t\in\mathbb{R}^{n_w} w t ∈ R n w
W t ∈ R n x × n w \mathbf{W}_t\in\mathbb{R}^{n_x\times n_w} W t ∈ R n x × n w w t \mathbf{w}_t w t
Q t = E [ w t w t ′ ] ∈ R n w × n w \mathbf{Q}_t=\mathbb{E}[\mathbf{w}_t\mathbf{w}_t^{\prime}]\in\mathbb{R}^{n_w\times n_w} Q t = E [ w t w t ′ ] ∈ R n w × n w w t \mathbf{w}_t w t
Measurement Equation
z t = H t x t + U t u t \mathbf{z}_t
=
\mathbf{H}_t\mathbf{x}_t
+
\mathbf{U}_t\mathbf{u}_t z t = H t x t + U t u t where:
z t ∈ R n z \mathbf{z}_t\in\mathbb{R}^{n_z} z t ∈ R n z
H t ∈ R n z × n x \mathbf{H}_t\in\mathbb{R}^{n_z\times n_x} H t ∈ R n z × n x x t \mathbf{x}_t x t z t \mathbf{z}_t z t
u t ∈ R n u \mathbf{u}_t\in\mathbb{R}^{n_u} u t ∈ R n u
U t ∈ R n z × n u \mathbf{U}_t\in\mathbb{R}^{n_z\times n_u} U t ∈ R n z × n u u t \mathbf{u}_t u t
R t = E [ u t u t ′ ] ∈ R n u × n u \mathbf{R}_t=\mathbb{E}[\mathbf{u}_t\mathbf{u}_t^{\prime}]\in\mathbb{R}^{n_u\times n_u} R t = E [ u t u t ′ ] ∈ R n u × n u u t \mathbf{u}_t u t
2. Algorithm
Kalman Filter proceeds in initialization and two recursive steps as follows:
(0) Initialization
x ^ 0 ∣ 0 = x 0 \widehat{\mathbf{x}}_{0\mid 0}=\mathbf{x}_0 x 0 ∣ 0 = x 0 P 0 ∣ 0 = P 0 \mathbf{P}_{0\mid 0}=\mathbf{P}_0 P 0 ∣ 0 = P 0 Recursive loop for t = 1 t=1 t = 1 N N N
(1) Prediction (Time Update)
Using all observations up to time t − 1 t-1 t − 1 t t t
p ( x t − 1 ∣ z 1 : t − 1 ) ⟶ p ( x t ∣ z 1 : t − 1 ) p\!\left(\mathbf{x}_{t-1}\mid\mathbf{z}_{1:t-1}\right)
\longrightarrow
p\!\left(\mathbf{x}_t\mid\mathbf{z}_{1:t-1}\right) p ( x t − 1 ∣ z 1 : t − 1 ) ⟶ p ( x t ∣ z 1 : t − 1 ) and specifically due to the Markov property and the zero mean of the process noise, we can estimate hidden state a priori by using the previous filtered estimate of the hidden state as:
x ^ t ∣ t − 1 ⏞ Prior conditional expectation of x t = E [ x t ∣ z 1 : t − 1 ] = E [ A t x t − 1 + W t w t ∣ z 1 : t − 1 ] = A t x ^ t − 1 ∣ t − 1 . \overbrace{\widehat{\mathbf{x}}_{t\mid t-1}}^{\substack{\text{Prior conditional}\\\text{expectation of }\mathbf{x}_t}}
=
\mathbb{E}\!\left[\mathbf{x}_t\mid\mathbf{z}_{1:t-1}\right]
=
\mathbb{E}\!\left[
\mathbf{A}_t\mathbf{x}_{t-1}
+
\mathbf{W}_t\mathbf{w}_t
\mid
\mathbf{z}_{1:t-1}
\right]
=
\mathbf{A}_t\widehat{\mathbf{x}}_{t-1\mid t-1}. x t ∣ t − 1 Prior conditional expectation of x t = E [ x t ∣ z 1 : t − 1 ] = E [ A t x t − 1 + W t w t ∣ z 1 : t − 1 ] = A t x t − 1 ∣ t − 1 . Then by considering estimation errors:
e t ∣ t − 1 = x t − x ^ t ∣ t − 1 \mathbf{e}_{t\mid t-1}
=
\mathbf{x}_t-
\widehat{\mathbf{x}}_{t\mid t-1} e t ∣ t − 1 = x t − x t ∣ t − 1 e t − 1 ∣ t − 1 = x t − 1 − x ^ t − 1 ∣ t − 1 \mathbf{e}_{t-1\mid t-1}
=
\mathbf{x}_{t-1}-
\widehat{\mathbf{x}}_{t-1\mid t-1} e t − 1 ∣ t − 1 = x t − 1 − x t − 1 ∣ t − 1 A priori error covariance, variance in case x t \mathbf{x}_t x t x ^ t ∣ t − 1 \widehat{\mathbf{x}}_{t\mid t-1} x t ∣ t − 1
P t ∣ t − 1 ⏞ Prior uncertainty of x ^ t ∣ t − 1 estimate = E [ ( x t − x ^ t ∣ t − 1 ) ( x t − x ^ t ∣ t − 1 ) ′ ] = E [ e t ∣ t − 1 e t ∣ t − 1 ′ ] ∈ R n x × n x . \overbrace{\mathbf{P}_{t\mid t-1}}^{\substack{\text{Prior uncertainty of}\\\widehat{\mathbf{x}}_{t\mid t-1}\text{ estimate}}}
=
\mathbb{E}\!\left[
\left(\mathbf{x}_t-\widehat{\mathbf{x}}_{t\mid t-1}\right)
\left(\mathbf{x}_t-\widehat{\mathbf{x}}_{t\mid t-1}\right)^{\prime}
\right]
=
\mathbb{E}\!\left[
\mathbf{e}_{t\mid t-1}\mathbf{e}_{t\mid t-1}^{\prime}
\right]
\in\mathbb{R}^{n_x\times n_x}. P t ∣ t − 1 Prior uncertainty of x t ∣ t − 1 estimate = E [ ( x t − x t ∣ t − 1 ) ( x t − x t ∣ t − 1 ) ′ ] = E [ e t ∣ t − 1 e t ∣ t − 1 ′ ] ∈ R n x × n x . This can be further written down as:
P t ∣ t − 1 = E [ ( A t x t − 1 + W t w t − A t x ^ t − 1 ∣ t − 1 ) ⋅ ( A t x t − 1 + W t w t − A t x ^ t − 1 ∣ t − 1 ) ′ ] = E [ ( A t ( x t − 1 − x ^ t − 1 ∣ t − 1 ) + W t w t ) ⋅ ( A t ( x t − 1 − x ^ t − 1 ∣ t − 1 ) + W t w t ) ′ ] = E [ ( A t e t − 1 ∣ t − 1 + W t w t ) ( A t e t − 1 ∣ t − 1 + W t w t ) ′ ] = E [ A t e t − 1 ∣ t − 1 e t − 1 ∣ t − 1 ′ A t ′ + A t e t − 1 ∣ t − 1 w t ′ W t ′ + W t w t e t − 1 ∣ t − 1 ′ A t ′ + W t w t w t ′ W t ′ ] = A t E [ e t − 1 ∣ t − 1 e t − 1 ∣ t − 1 ′ ] A t ′ + A t E [ e t − 1 ∣ t − 1 w t ′ ] W t ′ + W t E [ w t e t − 1 ∣ t − 1 ′ ] A t ′ + W t E [ w t w t ′ ] W t ′ . \begin{aligned}
\mathbf{P}_{t\mid t-1}
&=
\mathbb{E}\!\Big[
\big(
\mathbf{A}_t\mathbf{x}_{t-1}
+
\mathbf{W}_t\mathbf{w}_t
-
\mathbf{A}_t\widehat{\mathbf{x}}_{t-1\mid t-1}
\big)
\cdot
\big(
\mathbf{A}_t\mathbf{x}_{t-1}
+
\mathbf{W}_t\mathbf{w}_t
-
\mathbf{A}_t\widehat{\mathbf{x}}_{t-1\mid t-1}
\big)^{\prime}
\Big]
\\[0.4em]
&=
\mathbb{E}\!\Big[
\big(
\mathbf{A}_t(\mathbf{x}_{t-1}-\widehat{\mathbf{x}}_{t-1\mid t-1})
+
\mathbf{W}_t\mathbf{w}_t
\big)
\cdot
\big(
\mathbf{A}_t(\mathbf{x}_{t-1}-\widehat{\mathbf{x}}_{t-1\mid t-1})
+
\mathbf{W}_t\mathbf{w}_t
\big)^{\prime}
\Big]
\\[0.4em]
&=
\mathbb{E}\!\left[
\left(
\mathbf{A}_t\mathbf{e}_{t-1\mid t-1}
+
\mathbf{W}_t\mathbf{w}_t
\right)
\left(
\mathbf{A}_t\mathbf{e}_{t-1\mid t-1}
+
\mathbf{W}_t\mathbf{w}_t
\right)^{\prime}
\right]
\\[0.4em]
&=
\mathbb{E}\!\Big[
\mathbf{A}_t\mathbf{e}_{t-1\mid t-1}
\mathbf{e}_{t-1\mid t-1}^{\prime}\mathbf{A}_t^{\prime}
+
\mathbf{A}_t\mathbf{e}_{t-1\mid t-1}\mathbf{w}_t^{\prime}\mathbf{W}_t^{\prime}
+
\mathbf{W}_t\mathbf{w}_t\mathbf{e}_{t-1\mid t-1}^{\prime}\mathbf{A}_t^{\prime}
+
\mathbf{W}_t\mathbf{w}_t\mathbf{w}_t^{\prime}\mathbf{W}_t^{\prime}
\Big]
\\[0.4em]
&=
\mathbf{A}_t\mathbb{E}\!\left[
\mathbf{e}_{t-1\mid t-1}\mathbf{e}_{t-1\mid t-1}^{\prime}
\right]\mathbf{A}_t^{\prime}
+
\mathbf{A}_t\mathbb{E}\!\left[
\mathbf{e}_{t-1\mid t-1}\mathbf{w}_t^{\prime}
\right]\mathbf{W}_t^{\prime}
\\
&\quad+
\mathbf{W}_t\mathbb{E}\!\left[
\mathbf{w}_t\mathbf{e}_{t-1\mid t-1}^{\prime}
\right]\mathbf{A}_t^{\prime}
+
\mathbf{W}_t\mathbb{E}\!\left[
\mathbf{w}_t\mathbf{w}_t^{\prime}
\right]\mathbf{W}_t^{\prime}.
\end{aligned} P t ∣ t − 1 = E [ ( A t x t − 1 + W t w t − A t x t − 1 ∣ t − 1 ) ⋅ ( A t x t − 1 + W t w t − A t x t − 1 ∣ t − 1 ) ′ ] = E [ ( A t ( x t − 1 − x t − 1 ∣ t − 1 ) + W t w t ) ⋅ ( A t ( x t − 1 − x t − 1 ∣ t − 1 ) + W t w t ) ′ ] = E [ ( A t e t − 1 ∣ t − 1 + W t w t ) ( A t e t − 1 ∣ t − 1 + W t w t ) ′ ] = E [ A t e t − 1 ∣ t − 1 e t − 1 ∣ t − 1 ′ A t ′ + A t e t − 1 ∣ t − 1 w t ′ W t ′ + W t w t e t − 1 ∣ t − 1 ′ A t ′ + W t w t w t ′ W t ′ ] = A t E [ e t − 1 ∣ t − 1 e t − 1 ∣ t − 1 ′ ] A t ′ + A t E [ e t − 1 ∣ t − 1 w t ′ ] W t ′ + W t E [ w t e t − 1 ∣ t − 1 ′ ] A t ′ + W t E [ w t w t ′ ] W t ′ . where
P t − 1 ∣ t − 1 = E [ e t − 1 ∣ t − 1 e t − 1 ∣ t − 1 ′ ] , \mathbf{P}_{t-1\mid t-1}
=
\mathbb{E}\!\left[
\mathbf{e}_{t-1\mid t-1}\mathbf{e}_{t-1\mid t-1}^{\prime}
\right], P t − 1 ∣ t − 1 = E [ e t − 1 ∣ t − 1 e t − 1 ∣ t − 1 ′ ] , E [ e t − 1 ∣ t − 1 w t ′ ] = 0 , E [ w t e t − 1 ∣ t − 1 ′ ] = 0 , \mathbb{E}\!\left[
\mathbf{e}_{t-1\mid t-1}\mathbf{w}_t^{\prime}
\right]
=
\mathbf{0},
\qquad
\mathbb{E}\!\left[
\mathbf{w}_t\mathbf{e}_{t-1\mid t-1}^{\prime}
\right]
=
\mathbf{0}, E [ e t − 1 ∣ t − 1 w t ′ ] = 0 , E [ w t e t − 1 ∣ t − 1 ′ ] = 0 , and
Q t = E [ w t w t ′ ] , \mathbf{Q}_t
=
\mathbb{E}\!\left[
\mathbf{w}_t\mathbf{w}_t^{\prime}
\right], Q t = E [ w t w t ′ ] , so:
P t ∣ t − 1 ⏞ Prior uncertainty of x ^ t ∣ t − 1 estimate = A t P t − 1 ∣ t − 1 A t ′ ⏞ Uncertainty of x ^ t − 1 ∣ t − 1 estimate propagated by A t into the uncertainty of the predicted state at time t + W t Q t W t ′ ⏞ Uncertainty of w t propagated by W t into the uncertainty of the predicted state at time t . \begin{aligned}
\overbrace{\mathbf{P}_{t\mid t-1}}^{\substack{\text{Prior uncertainty of}\\\widehat{\mathbf{x}}_{t\mid t-1}\text{ estimate}}}
&=
\overbrace{
\mathbf{A}_t\mathbf{P}_{t-1\mid t-1}\mathbf{A}_t^{\prime}
}^{\substack{
\text{Uncertainty of }\widehat{\mathbf{x}}_{t-1\mid t-1}\text{ estimate}\\
\text{propagated by }\mathbf{A}_t\text{ into the uncertainty}\\
\text{of the predicted state at time }t
}}
+
\overbrace{
\mathbf{W}_t\mathbf{Q}_t\mathbf{W}_t^{\prime}
}^{\substack{
\text{Uncertainty of }\mathbf{w}_t\text{ propagated}\\
\text{by }\mathbf{W}_t\text{ into the uncertainty of}\\
\text{the predicted state at time }t
}}.
\end{aligned} P t ∣ t − 1 Prior uncertainty of x t ∣ t − 1 estimate = A t P t − 1 ∣ t − 1 A t ′ Uncertainty of x t − 1 ∣ t − 1 estimate propagated by A t into the uncertainty of the predicted state at time t + W t Q t W t ′ Uncertainty of w t propagated by W t into the uncertainty of the predicted state at time t . (2) Correction (Measurement Update)
After observing the new data point z t \mathbf{z}_t z t t t t
p ( x t ∣ z 1 : t − 1 ) ⟶ p ( x t ∣ z 1 : t ) p\!\left(\mathbf{x}_t\mid\mathbf{z}_{1:t-1}\right)
\longrightarrow
p\!\left(\mathbf{x}_t\mid\mathbf{z}_{1:t}\right) p ( x t ∣ z 1 : t − 1 ) ⟶ p ( x t ∣ z 1 : t ) By combining the previously predicted state estimate x ^ t ∣ t − 1 \widehat{\mathbf{x}}_{t\mid t-1} x t ∣ t − 1 z t \mathbf{z}_t z t
x ^ t ∣ t ⏞ Posterior conditional expectation of x t = E [ x t ∣ z 1 : t ] = x ^ t ∣ t − 1 + K t ν t = ( I − K t H t ) ⏞ Prior-state estimate weight adjusted by H t x ^ t ∣ t − 1 + K t ⏞ Measurement weight z t . \begin{aligned}
\overbrace{\widehat{\mathbf{x}}_{t\mid t}}^{\substack{\text{Posterior conditional}\\\text{expectation of }\mathbf{x}_t}}
&=
\mathbb{E}\!\left[\mathbf{x}_t\mid\mathbf{z}_{1:t}\right]
=
\widehat{\mathbf{x}}_{t\mid t-1}
+
\mathbf{K}_t\boldsymbol{\nu}_t
\\
&=
\overbrace{
\left(\mathbf{I}-\mathbf{K}_t\mathbf{H}_t\right)
}^{\substack{\text{Prior-state estimate}\\
\text{weight adjusted by }\mathbf{H}_t}}
\widehat{\mathbf{x}}_{t\mid t-1}
+
\overbrace{\mathbf{K}_t}^{\text{Measurement weight}}
\mathbf{z}_t.
\end{aligned} x t ∣ t Posterior conditional expectation of x t = E [ x t ∣ z 1 : t ] = x t ∣ t − 1 + K t ν t = ( I − K t H t ) Prior-state estimate weight adjusted by H t x t ∣ t − 1 + K t Measurement weight z t . where:
ν t \boldsymbol{\nu}_t ν t
ν t = z t − z ^ t ∣ t − 1 = H t x t + U t u t − H t x ^ t ∣ t − 1 ∈ R n z \boldsymbol{\nu}_t
=
\mathbf{z}_t-\widehat{\mathbf{z}}_{t\mid t-1}
=
\mathbf{H}_t\mathbf{x}_t+\mathbf{U}_t\mathbf{u}_t
-\mathbf{H}_t\widehat{\mathbf{x}}_{t\mid t-1}
\in\mathbb{R}^{n_z} ν t = z t − z t ∣ t − 1 = H t x t + U t u t − H t x t ∣ t − 1 ∈ R n z
K t \mathbf{K}_t K t ν t \boldsymbol{\nu}_t ν t K t \mathbf{K}_t K t P t ∣ t \mathbf{P}_{t\mid t} P t ∣ t
K t = (Incomplete) uncertainty of predicted observation Uncertainty of predicted observation + Uncertainty of measurement noise = P t ∣ t − 1 H t ′ C t ∣ t − 1 − 1 = P t ∣ t − 1 H t ′ ( H t P t ∣ t − 1 H t ′ + U t R t U t ′ ) − 1 ∈ R n x × n z \begin{aligned}
\mathbf{K}_t
&=
\frac{
\textit{\small (Incomplete) uncertainty of predicted observation}
}{
\textit{\small Uncertainty of predicted observation}
+
\textit{\small Uncertainty of measurement noise}
}
\\[0.5em]
&=
\mathbf{P}_{t\mid t-1}\mathbf{H}_t^{\prime}\mathbf{C}_{t\mid t-1}^{-1}
=
\mathbf{P}_{t\mid t-1}\mathbf{H}_t^{\prime}
\left(
\mathbf{H}_t\mathbf{P}_{t\mid t-1}\mathbf{H}_t^{\prime}
+
\mathbf{U}_t\mathbf{R}_t\mathbf{U}_t^{\prime}
\right)^{-1}
\in\mathbb{R}^{n_x\times n_z}
\end{aligned} K t = Uncertainty of predicted observation + Uncertainty of measurement noise (Incomplete) uncertainty of predicted observation = P t ∣ t − 1 H t ′ C t ∣ t − 1 − 1 = P t ∣ t − 1 H t ′ ( H t P t ∣ t − 1 H t ′ + U t R t U t ′ ) − 1 ∈ R n x × n z
C t ∣ t − 1 \mathbf{C}_{t\mid t-1} C t ∣ t − 1
C t ∣ t − 1 = E [ ν t ν t ′ ] = H t P t ∣ t − 1 H t ′ ⏟ Predicted-state uncertainty projected into observation space by H t + U t R t U t ′ ⏟ Measurement-noise uncertainty projected into observation space by U t ∈ R n z × n z . \mathbf{C}_{t\mid t-1}
=
\mathbb{E}\!\left[
\boldsymbol{\nu}_t\boldsymbol{\nu}_t^{\prime}
\right]
=
\underbrace{
\mathbf{H}_t\mathbf{P}_{t\mid t-1}\mathbf{H}_t^{\prime}
}_{\substack{
\text{Predicted-state uncertainty projected}\\
\text{into observation space by }\mathbf{H}_t
}}
+
\underbrace{
\mathbf{U}_t\mathbf{R}_t\mathbf{U}_t^{\prime}
}_{\substack{
\text{Measurement-noise uncertainty projected}\\
\text{into observation space by }\mathbf{U}_t
}}
\in\mathbb{R}^{n_z\times n_z}. C t ∣ t − 1 = E [ ν t ν t ′ ] = Predicted-state uncertainty projected into observation space by H t H t P t ∣ t − 1 H t ′ + Measurement-noise uncertainty projected into observation space by U t U t R t U t ′ ∈ R n z × n z .
P t ∣ t − 1 H t ′ \mathbf{P}_{t\mid t-1}\mathbf{H}_t^{\prime} P t ∣ t − 1 H t ′ H t ′ \mathbf{H}_t^{\prime} H t ′ H t \mathbf{H}_t H t I − K t H t \mathbf{I}-\mathbf{K}_t\mathbf{H}_t I − K t H t K t z t \mathbf{K}_t\mathbf{z}_t K t z t z t = H t x t + U t u t \mathbf{z}_t=\mathbf{H}_t\mathbf{x}_t+\mathbf{U}_t\mathbf{u}_t z t = H t x t + U t u t
Finally, the a posteriori error covariance, representing the uncertainty of the corrected hidden-state estimate x ^ t ∣ t \widehat{\mathbf{x}}_{t\mid t} x t ∣ t
P t ∣ t = ( I − K t H t ) P t ∣ t − 1 . \mathbf{P}_{t\mid t}
=
\left(
\mathbf{I}-\mathbf{K}_t\mathbf{H}_t
\right)
\mathbf{P}_{t\mid t-1}. P t ∣ t = ( I − K t H t ) P t ∣ t − 1 . Intuitively, we are correcting the predicted distribution using the information contained in the new observation z t \mathbf{z}_t z t
This formula can be derived as follows:
P t ∣ t = E [ e t ∣ t e t ∣ t ′ ] = E [ ( x t − x ^ t ∣ t ) ( x t − x ^ t ∣ t ) ′ ] = E [ ( x t − x ^ t ∣ t − 1 − K t ( z t − z ^ t ∣ t − 1 ) ) ⋅ ( x t − x ^ t ∣ t − 1 − K t ( z t − z ^ t ∣ t − 1 ) ) ′ ] = E [ ( x t − x ^ t ∣ t − 1 − K t ( H t x t + U t u t − H t x ^ t ∣ t − 1 ) ) ⋅ ( x t − x ^ t ∣ t − 1 − K t ( H t x t + U t u t − H t x ^ t ∣ t − 1 ) ) ′ ] = E [ ( ( I − K t H t ) ( x t − x ^ t ∣ t − 1 ) − K t U t u t ) ⋅ ( ( I − K t H t ) ( x t − x ^ t ∣ t − 1 ) − K t U t u t ) ′ ] = E [ ( ( I − K t H t ) e t ∣ t − 1 − K t U t u t ) ( ( I − K t H t ) e t ∣ t − 1 − K t U t u t ) ′ ] = ( I − K t H t ) E [ e t ∣ t − 1 e t ∣ t − 1 ′ ] ( I − K t H t ) ′ − ( I − K t H t ) E [ e t ∣ t − 1 u t ′ ] U t ′ K t ′ − K t U t E [ u t e t ∣ t − 1 ′ ] ( I − K t H t ) ′ + K t U t E [ u t u t ′ ] U t ′ K t ′ . \begin{aligned}
\mathbf{P}_{t\mid t}
&=
\mathbb{E}\!\left[
\mathbf{e}_{t\mid t}\mathbf{e}_{t\mid t}^{\prime}
\right]
\\[4pt]
&=
\mathbb{E}\!\left[
\left(
\mathbf{x}_t-\widehat{\mathbf{x}}_{t\mid t}
\right)
\left(
\mathbf{x}_t-\widehat{\mathbf{x}}_{t\mid t}
\right)^{\prime}
\right]
\\[4pt]
&=
\mathbb{E}\!\left[
\left(
\mathbf{x}_t
-
\widehat{\mathbf{x}}_{t\mid t-1}
-
\mathbf{K}_t
\left(
\mathbf{z}_t-\widehat{\mathbf{z}}_{t\mid t-1}
\right)
\right)\!\cdot\!
\left(
\mathbf{x}_t
-
\widehat{\mathbf{x}}_{t\mid t-1}
-
\mathbf{K}_t
\left(
\mathbf{z}_t-\widehat{\mathbf{z}}_{t\mid t-1}
\right)
\right)^{\prime}
\right]
\\[4pt]
&=
\mathbb{E}\!\left[
\left(
\mathbf{x}_t
-
\widehat{\mathbf{x}}_{t\mid t-1}
-
\mathbf{K}_t
\left(
\mathbf{H}_t\mathbf{x}_t
+
\mathbf{U}_t\mathbf{u}_t
-
\mathbf{H}_t\widehat{\mathbf{x}}_{t\mid t-1}
\right)
\right)\!\cdot\!
\left(
\mathbf{x}_t
-
\widehat{\mathbf{x}}_{t\mid t-1}
-
\mathbf{K}_t
\left(
\mathbf{H}_t\mathbf{x}_t
+
\mathbf{U}_t\mathbf{u}_t
-
\mathbf{H}_t\widehat{\mathbf{x}}_{t\mid t-1}
\right)
\right)^{\prime}
\right]
\\[4pt]
&=
\mathbb{E}\!\left[
\left(
\left(\mathbf{I}-\mathbf{K}_t\mathbf{H}_t\right)
\left(
\mathbf{x}_t-\widehat{\mathbf{x}}_{t\mid t-1}
\right)
-
\mathbf{K}_t\mathbf{U}_t\mathbf{u}_t
\right)
\!\cdot\!
\left(
\left(\mathbf{I}-\mathbf{K}_t\mathbf{H}_t\right)
\left(
\mathbf{x}_t-\widehat{\mathbf{x}}_{t\mid t-1}
\right)
-
\mathbf{K}_t\mathbf{U}_t\mathbf{u}_t
\right)^{\prime}
\right]
\\[4pt]
&=
\mathbb{E}\!\left[
\left(
\left(\mathbf{I}-\mathbf{K}_t\mathbf{H}_t\right)
\mathbf{e}_{t\mid t-1}
-
\mathbf{K}_t\mathbf{U}_t\mathbf{u}_t
\right)
\left(
\left(\mathbf{I}-\mathbf{K}_t\mathbf{H}_t\right)
\mathbf{e}_{t\mid t-1}
-
\mathbf{K}_t\mathbf{U}_t\mathbf{u}_t
\right)^{\prime}
\right]
\\[4pt]
&=
\left(\mathbf{I}-\mathbf{K}_t\mathbf{H}_t\right)
\mathbb{E}\!\left[
\mathbf{e}_{t\mid t-1}\mathbf{e}_{t\mid t-1}^{\prime}
\right]
\left(\mathbf{I}-\mathbf{K}_t\mathbf{H}_t\right)^{\prime}
\\[-2pt]
&\quad
-
\left(\mathbf{I}-\mathbf{K}_t\mathbf{H}_t\right)
\mathbb{E}\!\left[
\mathbf{e}_{t\mid t-1}\mathbf{u}_t^{\prime}
\right]
\mathbf{U}_t^{\prime}\mathbf{K}_t^{\prime}
\\[-2pt]
&\quad
-
\mathbf{K}_t\mathbf{U}_t
\mathbb{E}\!\left[
\mathbf{u}_t\mathbf{e}_{t\mid t-1}^{\prime}
\right]
\left(\mathbf{I}-\mathbf{K}_t\mathbf{H}_t\right)^{\prime}
\\[-2pt]
&\quad
+
\mathbf{K}_t\mathbf{U}_t
\mathbb{E}\!\left[
\mathbf{u}_t\mathbf{u}_t^{\prime}
\right]
\mathbf{U}_t^{\prime}\mathbf{K}_t^{\prime}.
\end{aligned} P t ∣ t = E [ e t ∣ t e t ∣ t ′ ] = E [ ( x t − x t ∣ t ) ( x t − x t ∣ t ) ′ ] = E [ ( x t − x t ∣ t − 1 − K t ( z t − z t ∣ t − 1 ) ) ⋅ ( x t − x t ∣ t − 1 − K t ( z t − z t ∣ t − 1 ) ) ′ ] = E [ ( x t − x t ∣ t − 1 − K t ( H t x t + U t u t − H t x t ∣ t − 1 ) ) ⋅ ( x t − x t ∣ t − 1 − K t ( H t x t + U t u t − H t x t ∣ t − 1 ) ) ′ ] = E [ ( ( I − K t H t ) ( x t − x t ∣ t − 1 ) − K t U t u t ) ⋅ ( ( I − K t H t ) ( x t − x t ∣ t − 1 ) − K t U t u t ) ′ ] = E [ ( ( I − K t H t ) e t ∣ t − 1 − K t U t u t ) ( ( I − K t H t ) e t ∣ t − 1 − K t U t u t ) ′ ] = ( I − K t H t ) E [ e t ∣ t − 1 e t ∣ t − 1 ′ ] ( I − K t H t ) ′ − ( I − K t H t ) E [ e t ∣ t − 1 u t ′ ] U t ′ K t ′ − K t U t E [ u t e t ∣ t − 1 ′ ] ( I − K t H t ) ′ + K t U t E [ u t u t ′ ] U t ′ K t ′ . And because we know that:
E [ e t ∣ t − 1 u t ′ ] = 0 , E [ u t e t ∣ t − 1 ′ ] = 0 , \mathbb{E}\!\left[
\mathbf{e}_{t\mid t-1}\mathbf{u}_t^{\prime}
\right]
=
\mathbf{0},
\qquad
\mathbb{E}\!\left[
\mathbf{u}_t\mathbf{e}_{t\mid t-1}^{\prime}
\right]
=
\mathbf{0}, E [ e t ∣ t − 1 u t ′ ] = 0 , E [ u t e t ∣ t − 1 ′ ] = 0 , we can continue as follows:
P t ∣ t = ( I − K t H t ) P t ∣ t − 1 ( I − K t H t ) ′ + K t U t R t U t ′ K t ′ = ( I − K t H t ) P t ∣ t − 1 ( I − H t ′ K t ′ ) + K t U t R t U t ′ K t ′ = P t ∣ t − 1 ( I − H t ′ K t ′ ) − K t H t P t ∣ t − 1 ( I − H t ′ K t ′ ) + K t U t R t U t ′ K t ′ = P t ∣ t − 1 − P t ∣ t − 1 H t ′ K t ′ − K t H t P t ∣ t − 1 + K t H t P t ∣ t − 1 H t ′ K t ′ + K t U t R t U t ′ K t ′ = P t ∣ t − 1 − K t H t P t ∣ t − 1 − P t ∣ t − 1 H t ′ K t ′ + K t ( H t P t ∣ t − 1 H t ′ + U t R t U t ′ ) K t ′ = P t ∣ t − 1 − K t H t P t ∣ t − 1 − P t ∣ t − 1 H t ′ K t ′ + K t C t ∣ t − 1 K t ′ = P t ∣ t − 1 − K t H t P t ∣ t − 1 − P t ∣ t − 1 H t ′ K t ′ + ( P t ∣ t − 1 H t ′ C t ∣ t − 1 − 1 ) C t ∣ t − 1 K t ′ = P t ∣ t − 1 − K t H t P t ∣ t − 1 − P t ∣ t − 1 H t ′ K t ′ + P t ∣ t − 1 H t ′ K t ′ = P t ∣ t − 1 − K t H t P t ∣ t − 1 = ( I − K t H t ) P t ∣ t − 1 . \begin{aligned}
\mathbf{P}_{t\mid t}
&=
\left(\mathbf{I}-\mathbf{K}_t\mathbf{H}_t\right)
\mathbf{P}_{t\mid t-1}
\left(\mathbf{I}-\mathbf{K}_t\mathbf{H}_t\right)^{\prime}
+
\mathbf{K}_t\mathbf{U}_t\mathbf{R}_t\mathbf{U}_t^{\prime}\mathbf{K}_t^{\prime}
\\[4pt]
&=
\left(\mathbf{I}-\mathbf{K}_t\mathbf{H}_t\right)
\mathbf{P}_{t\mid t-1}
\left(\mathbf{I}-\mathbf{H}_t^{\prime}\mathbf{K}_t^{\prime}\right)
+
\mathbf{K}_t\mathbf{U}_t\mathbf{R}_t\mathbf{U}_t^{\prime}\mathbf{K}_t^{\prime}
\\[4pt]
&=
\mathbf{P}_{t\mid t-1}
\left(\mathbf{I}-\mathbf{H}_t^{\prime}\mathbf{K}_t^{\prime}\right)
-
\mathbf{K}_t\mathbf{H}_t\mathbf{P}_{t\mid t-1}
\left(\mathbf{I}-\mathbf{H}_t^{\prime}\mathbf{K}_t^{\prime}\right)
+
\mathbf{K}_t\mathbf{U}_t\mathbf{R}_t\mathbf{U}_t^{\prime}\mathbf{K}_t^{\prime}
\\[4pt]
&=
\mathbf{P}_{t\mid t-1}
-
\mathbf{P}_{t\mid t-1}\mathbf{H}_t^{\prime}\mathbf{K}_t^{\prime}
-
\mathbf{K}_t\mathbf{H}_t\mathbf{P}_{t\mid t-1}
+
\mathbf{K}_t\mathbf{H}_t
\mathbf{P}_{t\mid t-1}
\mathbf{H}_t^{\prime}\mathbf{K}_t^{\prime}
+
\mathbf{K}_t\mathbf{U}_t\mathbf{R}_t\mathbf{U}_t^{\prime}\mathbf{K}_t^{\prime}
\\[4pt]
&=
\mathbf{P}_{t\mid t-1}
-
\mathbf{K}_t\mathbf{H}_t\mathbf{P}_{t\mid t-1}
-
\mathbf{P}_{t\mid t-1}\mathbf{H}_t^{\prime}\mathbf{K}_t^{\prime}
+
\mathbf{K}_t
\left(
\mathbf{H}_t\mathbf{P}_{t\mid t-1}\mathbf{H}_t^{\prime}
+
\mathbf{U}_t\mathbf{R}_t\mathbf{U}_t^{\prime}
\right)
\mathbf{K}_t^{\prime}
\\[4pt]
&=
\mathbf{P}_{t\mid t-1}
-
\mathbf{K}_t\mathbf{H}_t\mathbf{P}_{t\mid t-1}
-
\mathbf{P}_{t\mid t-1}\mathbf{H}_t^{\prime}\mathbf{K}_t^{\prime}
+
\mathbf{K}_t\mathbf{C}_{t\mid t-1}\mathbf{K}_t^{\prime}
\\[4pt]
&=
\mathbf{P}_{t\mid t-1}
-
\mathbf{K}_t\mathbf{H}_t\mathbf{P}_{t\mid t-1}
-
\mathbf{P}_{t\mid t-1}\mathbf{H}_t^{\prime}\mathbf{K}_t^{\prime}
+
\left(
\mathbf{P}_{t\mid t-1}\mathbf{H}_t^{\prime}\mathbf{C}_{t\mid t-1}^{-1}
\right)
\mathbf{C}_{t\mid t-1}\mathbf{K}_t^{\prime}
\\[4pt]
&=
\mathbf{P}_{t\mid t-1}
-
\mathbf{K}_t\mathbf{H}_t\mathbf{P}_{t\mid t-1}
-
\mathbf{P}_{t\mid t-1}\mathbf{H}_t^{\prime}\mathbf{K}_t^{\prime}
+
\mathbf{P}_{t\mid t-1}\mathbf{H}_t^{\prime}\mathbf{K}_t^{\prime}
\\[4pt]
&=
\mathbf{P}_{t\mid t-1}
-
\mathbf{K}_t\mathbf{H}_t\mathbf{P}_{t\mid t-1}
\\[4pt]
&=
\left(
\mathbf{I}-\mathbf{K}_t\mathbf{H}_t
\right)
\mathbf{P}_{t\mid t-1}.
\end{aligned} P t ∣ t = ( I − K t H t ) P t ∣ t − 1 ( I − K t H t ) ′ + K t U t R t U t ′ K t ′ = ( I − K t H t ) P t ∣ t − 1 ( I − H t ′ K t ′ ) + K t U t R t U t ′ K t ′ = P t ∣ t − 1 ( I − H t ′ K t ′ ) − K t H t P t ∣ t − 1 ( I − H t ′ K t ′ ) + K t U t R t U t ′ K t ′ = P t ∣ t − 1 − P t ∣ t − 1 H t ′ K t ′ − K t H t P t ∣ t − 1 + K t H t P t ∣ t − 1 H t ′ K t ′ + K t U t R t U t ′ K t ′ = P t ∣ t − 1 − K t H t P t ∣ t − 1 − P t ∣ t − 1 H t ′ K t ′ + K t ( H t P t ∣ t − 1 H t ′ + U t R t U t ′ ) K t ′ = P t ∣ t − 1 − K t H t P t ∣ t − 1 − P t ∣ t − 1 H t ′ K t ′ + K t C t ∣ t − 1 K t ′ = P t ∣ t − 1 − K t H t P t ∣ t − 1 − P t ∣ t − 1 H t ′ K t ′ + ( P t ∣ t − 1 H t ′ C t ∣ t − 1 − 1 ) C t ∣ t − 1 K t ′ = P t ∣ t − 1 − K t H t P t ∣ t − 1 − P t ∣ t − 1 H t ′ K t ′ + P t ∣ t − 1 H t ′ K t ′ = P t ∣ t − 1 − K t H t P t ∣ t − 1 = ( I − K t H t ) P t ∣ t − 1 . 3. Parameter Estimation with MLE
The parameter set
θ = { A t , W t , H t , U t } \boldsymbol{\theta}
=
\left\{
\mathbf{A}_t,
\mathbf{W}_t,
\mathbf{H}_t,
\mathbf{U}_t
\right\} θ = { A t , W t , H t , U t } can be estimated using Maximum Likelihood Estimation by finding the parameter values under which the observed sequence z 1 , … , z N \mathbf{z}_1,\ldots,\mathbf{z}_N z 1 , … , z N z t \mathbf{z}_t z t
θ ^ = arg max θ p ( z 1 : N ∣ θ ) = arg max θ ∏ t = 1 N p ( z t ∣ z 1 : t − 1 , θ ) . \widehat{\boldsymbol{\theta}}
=
\underset{\boldsymbol{\theta}}{\arg\max}
\;
p\!\left(
\mathbf{z}_{1:N}
\mid
\boldsymbol{\theta}
\right)
=
\underset{\boldsymbol{\theta}}{\arg\max}
\;
\prod_{t=1}^{N}
p\!\left(
\mathbf{z}_t
\mid
\mathbf{z}_{1:t-1},
\boldsymbol{\theta}
\right). θ = θ arg max p ( z 1 : N ∣ θ ) = θ arg max t = 1 ∏ N p ( z t ∣ z 1 : t − 1 , θ ) . As established in the previous section, the Kalman Filter provides the following one-step-ahead conditional mean and covariance matrix for the measurement variable z t \mathbf{z}_t z t
z ^ t ∣ t − 1 = E [ z t ∣ z 1 : t − 1 ] = E [ H t x t + U t u t ∣ z 1 : t − 1 ] = H t x ^ t ∣ t − 1 , \widehat{\mathbf{z}}_{t\mid t-1}
=
\mathbb{E}\!\left[
\mathbf{z}_t
\mid
\mathbf{z}_{1:t-1}
\right]
=
\mathbb{E}\!\left[
\mathbf{H}_t\mathbf{x}_t
+
\mathbf{U}_t\mathbf{u}_t
\mid
\mathbf{z}_{1:t-1}
\right]
=
\mathbf{H}_t
\widehat{\mathbf{x}}_{t\mid t-1}, z t ∣ t − 1 = E [ z t ∣ z 1 : t − 1 ] = E [ H t x t + U t u t ∣ z 1 : t − 1 ] = H t x t ∣ t − 1 , C t ∣ t − 1 = E [ ( z t − z ^ t ∣ t − 1 ) ( z t − z ^ t ∣ t − 1 ) ′ ∣ z 1 : t − 1 ] = H t P t ∣ t − 1 H t ′ + U t R t U t ′ , \mathbf{C}_{t\mid t-1}
=
\mathbb{E}\!\left[
\left(
\mathbf{z}_t-\widehat{\mathbf{z}}_{t\mid t-1}
\right)
\left(
\mathbf{z}_t-\widehat{\mathbf{z}}_{t\mid t-1}
\right)^{\prime}
\mid
\mathbf{z}_{1:t-1}
\right]
=
\mathbf{H}_t
\mathbf{P}_{t\mid t-1}
\mathbf{H}_t^{\prime}
+
\mathbf{U}_t
\mathbf{R}_t
\mathbf{U}_t^{\prime}, C t ∣ t − 1 = E [ ( z t − z t ∣ t − 1 ) ( z t − z t ∣ t − 1 ) ′ ∣ z 1 : t − 1 ] = H t P t ∣ t − 1 H t ′ + U t R t U t ′ , which fully characterize the one-step-ahead Gaussian predictive distribution of z t \mathbf{z}_t z t z 1 : t − 1 \mathbf{z}_{1:t-1} z 1 : t − 1 θ \boldsymbol{\theta} θ
z t ∣ z 1 : t − 1 , θ ∼ N ( z ^ t ∣ t − 1 , C t ∣ t − 1 ) . \mathbf{z}_t
\mid
\mathbf{z}_{1:t-1},
\boldsymbol{\theta}
\sim
\mathcal{N}\!\left(
\widehat{\mathbf{z}}_{t\mid t-1},
\mathbf{C}_{t\mid t-1}
\right). z t ∣ z 1 : t − 1 , θ ∼ N ( z t ∣ t − 1 , C t ∣ t − 1 ) . Then, the corresponding conditional probability density function evaluated at the observed value z t \mathbf{z}_t z t
p ( z t ∣ z 1 : t − 1 , θ ) = 1 ( 2 π ) n z det ( C t ∣ t − 1 ) exp [ − 1 2 ( z t − z ^ t ∣ t − 1 ) ′ C t ∣ t − 1 − 1 ( z t − z ^ t ∣ t − 1 ) ] . p\!\left(
\mathbf{z}_t
\mid
\mathbf{z}_{1:t-1},
\boldsymbol{\theta}
\right)
=
\frac{1}{
\sqrt{
(2\pi)^{n_z}
\det\!\left(\mathbf{C}_{t\mid t-1}\right)
}
}
\exp\!\left[
-\frac{1}{2}
\left(
\mathbf{z}_t-\widehat{\mathbf{z}}_{t\mid t-1}
\right)^{\prime}
\mathbf{C}_{t\mid t-1}^{-1}
\left(
\mathbf{z}_t-\widehat{\mathbf{z}}_{t\mid t-1}
\right)
\right]. p ( z t ∣ z 1 : t − 1 , θ ) = ( 2 π ) n z det ( C t ∣ t − 1 ) 1 exp [ − 2 1 ( z t − z t ∣ t − 1 ) ′ C t ∣ t − 1 − 1 ( z t − z t ∣ t − 1 ) ] . where n z n_z n z z t \mathbf{z}_t z t N N N
Substituting this conditional Normal density into the likelihood of the complete observed sequence gives:
L 1 : N ( θ ) = ∏ t = 1 N p ( z t ∣ z 1 : t − 1 , θ ) = ∏ t = 1 N [ 1 ( 2 π ) n z det ( C t ∣ t − 1 ) exp [ − 1 2 ( z t − z ^ t ∣ t − 1 ) ′ C t ∣ t − 1 − 1 ( z t − z ^ t ∣ t − 1 ) ] ] . \begin{aligned}
\mathcal{L}_{1:N}\!\left(\boldsymbol{\theta}\right)
&=
\prod_{t=1}^{N}
p\!\left(
\mathbf{z}_t
\mid
\mathbf{z}_{1:t-1},
\boldsymbol{\theta}
\right)
\\[4pt]
&=
\prod_{t=1}^{N}
\left[
\frac{1}{
\sqrt{
(2\pi)^{n_z}
\det\!\left(\mathbf{C}_{t\mid t-1}\right)
}
}
\exp\!\left[
-\frac{1}{2}
\left(
\mathbf{z}_t-\widehat{\mathbf{z}}_{t\mid t-1}
\right)^{\prime}
\mathbf{C}_{t\mid t-1}^{-1}
\left(
\mathbf{z}_t-\widehat{\mathbf{z}}_{t\mid t-1}
\right)
\right]
\right].
\end{aligned} L 1 : N ( θ ) = t = 1 ∏ N p ( z t ∣ z 1 : t − 1 , θ ) = t = 1 ∏ N ( 2 π ) n z det ( C t ∣ t − 1 ) 1 exp [ − 2 1 ( z t − z t ∣ t − 1 ) ′ C t ∣ t − 1 − 1 ( z t − z t ∣ t − 1 ) ] . For optimization, the likelihood can be simplified by taking its logarithm, which transforms the product over time into a sum:
ln L 1 : N ( θ ) = ∑ t = 1 N ln [ 1 ( 2 π ) n z det ( C t ∣ t − 1 ) exp [ − 1 2 ( z t − z ^ t ∣ t − 1 ) ′ C t ∣ t − 1 − 1 ( z t − z ^ t ∣ t − 1 ) ] ] = − 1 2 ∑ t = 1 N [ n z ln ( 2 π ) + ln det ( C t ∣ t − 1 ) + ( z t − z ^ t ∣ t − 1 ) ′ C t ∣ t − 1 − 1 ( z t − z ^ t ∣ t − 1 ) ] . \begin{aligned}
\ln\mathcal{L}_{1:N}\!\left(\boldsymbol{\theta}\right)
&=
\sum_{t=1}^{N}
\ln\!\left[
\frac{1}{
\sqrt{
(2\pi)^{n_z}
\det\!\left(\mathbf{C}_{t\mid t-1}\right)
}
}
\exp\!\left[
-\frac{1}{2}
\left(
\mathbf{z}_t-\widehat{\mathbf{z}}_{t\mid t-1}
\right)^{\prime}
\mathbf{C}_{t\mid t-1}^{-1}
\left(
\mathbf{z}_t-\widehat{\mathbf{z}}_{t\mid t-1}
\right)
\right]
\right]
\\[4pt]
&=
-\frac{1}{2}
\sum_{t=1}^{N}
\left[
n_z\ln(2\pi)
+
\ln\det\!\left(\mathbf{C}_{t\mid t-1}\right)
+
\left(
\mathbf{z}_t-\widehat{\mathbf{z}}_{t\mid t-1}
\right)^{\prime}
\mathbf{C}_{t\mid t-1}^{-1}
\left(
\mathbf{z}_t-\widehat{\mathbf{z}}_{t\mid t-1}
\right)
\right].
\end{aligned} ln L 1 : N ( θ ) = t = 1 ∑ N ln ( 2 π ) n z det ( C t ∣ t − 1 ) 1 exp [ − 2 1 ( z t − z t ∣ t − 1 ) ′ C t ∣ t − 1 − 1 ( z t − z t ∣ t − 1 ) ] = − 2 1 t = 1 ∑ N [ n z ln ( 2 π ) + ln det ( C t ∣ t − 1 ) + ( z t − z t ∣ t − 1 ) ′ C t ∣ t − 1 − 1 ( z t − z t ∣ t − 1 ) ] . This expression can be simplified further. Since the term n z ln ( 2 π ) n_z\ln(2\pi) n z ln ( 2 π ) θ \boldsymbol{\theta} θ − 1 / 2 -1/2 − 1/2
θ ^ = arg min θ L 1 : N ( θ ) = arg min θ ∑ t = 1 N [ ln det ( C t ∣ t − 1 ) ⏟ Prevention from making every observation unsurprising + ( z t − z ^ t ∣ t − 1 ) ′ C t ∣ t − 1 − 1 ( z t − z ^ t ∣ t − 1 ) ⏟ Penalization of prediction error ] . \begin{aligned}
\widehat{\boldsymbol{\theta}}
&=
\underset{\boldsymbol{\theta}}{\arg\min}
\;
L_{1:N}\!\left(\boldsymbol{\theta}\right)
\\[4pt]
&=
\underset{\boldsymbol{\theta}}{\arg\min}
\;
\sum_{t=1}^{N}
\left[
\underbrace{
\ln\det\!\left(\mathbf{C}_{t\mid t-1}\right)
}_{\substack{
\text{Prevention from making}\\
\text{every observation unsurprising}
}}
+
\underbrace{
\left(
\mathbf{z}_t-\widehat{\mathbf{z}}_{t\mid t-1}
\right)^{\prime}
\mathbf{C}_{t\mid t-1}^{-1}
\left(
\mathbf{z}_t-\widehat{\mathbf{z}}_{t\mid t-1}
\right)
}_{\substack{
\text{Penalization of}\\
\text{prediction error}
}}
\right].
\end{aligned} θ = θ arg min L 1 : N ( θ ) = θ arg min t = 1 ∑ N Prevention from making every observation unsurprising ln det ( C t ∣ t − 1 ) + Penalization of prediction error ( z t − z t ∣ t − 1 ) ′ C t ∣ t − 1 − 1 ( z t − z t ∣ t − 1 ) . When z t z_t z t C t ∣ t − 1 C_{t\mid t-1} C t ∣ t − 1 1 / C t ∣ t − 1 1/C_{t\mid t-1} 1/ C t ∣ t − 1
L 1 : N ( θ ) = ∑ t = 1 N [ ln ( C t ∣ t − 1 ) + ( z t − z ^ t ∣ t − 1 ) 2 C t ∣ t − 1 ] . L_{1:N}(\theta)
=
\sum_{t=1}^{N}
\left[
\ln\!\left(C_{t\mid t-1}\right)
+
\frac{
\left(
z_t-\widehat{z}_{t\mid t-1}
\right)^2
}{
C_{t\mid t-1}
}
\right]. L 1 : N ( θ ) = t = 1 ∑ N [ ln ( C t ∣ t − 1 ) + C t ∣ t − 1 ( z t − z t ∣ t − 1 ) 2 ] .