Variance components in errors-in-variables models: estimability, stability and bias analysis

Abstract

Although total least squares has been substantially investigated theoretically and widely applied in practical applications, almost nothing has been done to simultaneously address the estimation of parameters and the errors-in-variables (EIV) stochastic model. We prove that the variance components of the EIV stochastic model are not estimable, if the elements of the random coefficient matrix can be classified into two or more groups of data of the same accuracy. This result of inestimability is surprising as it indicates that we have no way of gaining any knowledge on such an EIV stochastic model. We demonstrate that the linear equations for the estimation of variance components could be ill-conditioned, if the variance components are theoretically estimable. Finally, if the variance components are estimable, we derive the biases of their estimates, which could be significantly amplified due to a large condition number.

Appendix: the matrices required for the MINQUE estimation of variance components

To apply the MINQUE method to estimate the variance components of (2) in association with the linearized EIV model (10), with the design matrix $\mathbf {A}(\hat{{\varvec{\beta }}},\hat{\overline{\mathbf {a}}})$ in (17), we have to preparatorily compute the normal, the projection and other necessary matrices. We should note that all these matrices, including any quantity computed with these matrices, depend on the estimates $\hat{{\varvec{\beta }}}$ and $\hat{\overline{\mathbf {a}}}$. For conciseness of notations, we will not write down such dependence without confusion. By definition, the normal matrix, denoted by $\mathbf {N}$, is given as follows:

$$\begin{aligned} \mathbf {N}&= \left[ \begin{array}{c@{\quad }c} (\hat{\overline{\mathbf {A}}})^T &{} \mathbf {0} \\ (\hat{{\varvec{\beta }}}\otimes \mathbf {I}_n) &{} \mathbf {I}_a \end{array} \right] \left[ \begin{array}{c@{\quad }c} {\varvec{\Sigma }}_{0y}^{-1} &{} \mathbf {0} \\ \mathbf {0} &{} {\varvec{\Sigma }}_{0a}^{-1} \end{array} \right] \left[ \begin{array}{l@{\quad }l} \hat{\overline{\mathbf {A}}} &{} {(\hat{{\varvec{\beta }}})^T\otimes \mathbf {I}_n}\\ \mathbf {0} &{} \mathbf {I}_a \end{array} \right] \nonumber \\&= \left[ \begin{array}{c@{\quad }c} (\hat{\overline{\mathbf {A}}})^T{\varvec{\Sigma }}_{0y}^{-1}\hat{ \overline{\mathbf {A}}} &{} (\hat{{\varvec{\beta }}})^T\otimes {(\hat{\overline{\mathbf {A}}})^T{\varvec{\Sigma }}_{0y}^{-1}} \\ \hat{{\varvec{\beta }}}\otimes ({\varvec{\Sigma }}_{0y}^{-1}\hat{ \overline{\mathbf {A}}}) &{} {\varvec{\Sigma }}_{0a}^{-1} + {\hat{{\varvec{\beta }}}(\hat{{\varvec{\beta }}})^T} \otimes {\varvec{\Sigma }}_{0y}^{-1} \end{array} \right] \nonumber \\&= \left[ \begin{array}{l@{\quad }l} \mathbf {N}_{\beta } &{} \mathbf {N}_{\beta \overline{a}} \\ \mathbf {N}_{\overline{a}\beta } &{} \mathbf {N}_{\overline{a}} \end{array} \right] . \end{aligned}$$

(48a)

The inverse of the normal matrix $\mathbf {N}$ with the four blocks is symbolically represented by the matrix $\mathbf {Q}$, namely,

$$\begin{aligned} \mathbf {Q} = \mathbf {N}^{-1} = \left[ \begin{array}{cc} \mathbf {Q}_{\beta } &{} \mathbf {Q}_{\beta \overline{a}} \\ \mathbf {Q}_{\overline{a}\beta } &{} \mathbf {Q}_{\overline{a}} \end{array} \right] , \end{aligned}$$

(48b)

with the size of each of the blocks $\mathbf {Q}_{\beta }$, $\mathbf {Q}_{\beta \overline{a}}$, $\mathbf {Q}_{\overline{a}\beta }$ and $\mathbf {Q}_{\overline{a}}$ corresponding to that of $\mathbf {N}_{\beta }$, $\mathbf {N}_{ \beta \overline{a}}$, $\mathbf {N}_{\overline{a}\beta }$ and $\mathbf {N}_{\overline{a}}$, respectively.

To compute the projection matrix, we first compute the following $\mathbf {H}$ matrix:

$$\begin{aligned} \mathbf {H}&= \left[ \begin{array}{l@{\quad }l} \hat{\overline{\mathbf {A}}} &{} {( \hat{{\varvec{\beta }}})^T\otimes \mathbf {I}_n} \\ \mathbf {0} &{} \mathbf {I}_a \end{array} \right] \left[ \begin{array}{cc} \mathbf {Q}_{\beta } &{} \mathbf {Q}_{\beta \overline{a}} \\ \mathbf {Q}_{\overline{a}\beta } &{} \mathbf {Q}_{\overline{a}} \end{array} \right] \left[ \begin{array}{l@{\quad }l} (\hat{\overline{\mathbf {A}}})^T &{} \mathbf {0} \\ (\hat{{\varvec{\beta }}}\otimes \mathbf {I}_n) &{} \mathbf {I}_a \end{array} \right] \nonumber \\&= \left[ \begin{array}{c@{\quad }c} \mathbf {H}_y &{} \mathbf {H}_{ya} \\ \mathbf {H}_{ay} &{} \mathbf {H}_a \end{array} \right] , \end{aligned}$$

(48c)

or explicitly by

$$\begin{aligned}&\mathbf {H}_y = \hat{\overline{\mathbf {A}}} \mathbf {Q}_{\beta }(\hat{\overline{\mathbf {A}}})^T +\hat{\overline{\mathbf {A}}}\mathbf {Q}_{\beta \overline{a}} (\hat{{\varvec{\beta }}}\otimes \mathbf {I}_n) + {(\hat{{\varvec{\beta }}})^T\otimes \mathbf {I}_n}\mathbf {Q}_{\overline{a}\beta }(\hat{\overline{\mathbf {A}}})^T\\&\qquad + {(\hat{{\varvec{\beta }}})^T\otimes \mathbf {I}_n}\mathbf {Q}_{\overline{a}}(\hat{{\varvec{\beta }}}\otimes \mathbf {I}_n),\\&\mathbf {H}_{ya} = \hat{\overline{\mathbf {A}}}\mathbf {Q}_{\beta \overline{a}} + {(\hat{{\varvec{\beta }}})^T\otimes \mathbf {I}_n}\mathbf {Q}_{\overline{a}},\\&\mathbf {H}_{ay} = \mathbf {H}_{ya}^T =\mathbf {Q}_{\overline{a}\beta }(\hat{\overline{\mathbf {A}}})^T+\mathbf {Q}_{\overline{a}}(\hat{{\varvec{\beta }}}\otimes \mathbf {I}_n),\\&\mathbf {H}_a = \mathbf {Q}_{\overline{a}}. \end{aligned}$$

With (14) in mind, the projection matrix in a linear model is defined by $\mathbf {R}=\mathbf {Z}\!_A^{\bot }=\mathbf {I}_{ya}-\mathbf {Z}\!_A$, where $\mathbf {I}_{ya}$ is an identity matrix with the dimension of sizes $\mathbf {y}$ and $\mathbf {A}$. As a result, we can finally obtain the projection matrix as follows:

$$\begin{aligned} \mathbf {R}&= \mathbf {I}_{ya} - \mathbf {H}{\varvec{\Sigma }}_0^{-1} \nonumber \\&= \left[ \begin{array}{cc} \mathbf {I}_n &{} \mathbf {0} \\ \mathbf {0} &{} \mathbf {I}_a \end{array} \right] - \mathbf {H}\left[ \begin{array}{cc} {\varvec{\Sigma }}_{0y}^{-1} &{} \mathbf {0} \\ \mathbf {0} &{} {\varvec{\Sigma }}_{0a}^{-1} \end{array} \right] \nonumber \\&= \left[ \begin{array}{cc} \mathbf {I}_n - \mathbf {H}_y{\varvec{\Sigma }}_{0y}^{-1} &{} -\mathbf {H}_{ya}{\varvec{\Sigma }}_{0a}^{-1} \\ -\mathbf {H}_{ay}{\varvec{\Sigma }}_{0y}^{-1} &{} \mathbf {I}_a - \mathbf {H}_a{\varvec{\Sigma }}_{0a}^{-1} \end{array} \right] \nonumber \\&= \left[ \begin{array}{cc} \mathbf {R}_y &{} \mathbf {R}_{ya} \\ \mathbf {R}_{ay} &{} \mathbf {R}_a \end{array} \right] . \end{aligned}$$

(49)

Finally, the matrix $\mathbf {P}$ in Sect. 3.2 is given as follows:

$$\begin{aligned} \mathbf {P}&= {\varvec{\Sigma }}_{0}^{-1}\mathbf {R} \nonumber \\&= \left[ \begin{array}{cc} {\varvec{\Sigma }}_{0y}^{-1} &{} \mathbf {0} \\ \mathbf {0} &{} {\varvec{\Sigma }}_{0a}^{-1} \end{array} \right] \left[ \begin{array}{cc} \mathbf {I}_n - \mathbf {H}_y{\varvec{\Sigma }}_{0y}^{-1} &{} -\mathbf {H}_{ya}{\varvec{\Sigma }}_{0a}^{-1} \\ -\mathbf {H}_{ay}{\varvec{\Sigma }}_{0y}^{-1} &{} \mathbf {I}_a - \mathbf {H}_a{\varvec{\Sigma }}_{0a}^{-1} \end{array} \right] \nonumber \\&= \left[ \begin{array}{cc} {\varvec{\Sigma }}_{0y}^{-1} - {\varvec{\Sigma }}_{0y}^{-1}\mathbf {H}_y{\varvec{ \Sigma }}_{0y}^{-1} &{} -{\varvec{\Sigma }}_{0y}^{-1}\mathbf {H}_{ya}{\varvec{\Sigma }}_{0a}^{-1} \\ -{\varvec{\Sigma }}_{0a}^{-1}\mathbf {H}_{ay}{\varvec{\Sigma }}_{0y}^{-1} &{} {\varvec{\Sigma }}_{0a}^{-1} - {\varvec{\Sigma }}_{0a}^{-1}\mathbf {H}_a{\varvec{\Sigma }}_{0a}^{-1} \end{array} \right] \nonumber \\&= \left[ \begin{array}{cc} \mathbf {P}_y &{} \mathbf {P}_{ya} \\ \mathbf {P}_{ay} &{} \mathbf {P}_a \end{array} \right] . \end{aligned}$$

(50)