Information matrix test

In econometrics, the information matrix test is used to determine whether a regression model is misspecified. The test was developed by Halbert White, ^[1] who observed that in a correctly specified model and under standard regularity assumptions, the Fisher information matrix can be expressed in either of two ways: as the outer product of the gradient of the log-likelihood function, or as a function of its Hessian matrix.

Consider a linear model ${\displaystyle \mathbf {y} =\mathbf {X} \mathbf {\beta } +\mathbf {u} }$, where the errors ${\displaystyle \mathbf {u} }$ are assumed to be distributed ${\displaystyle \mathrm {N} (0,\sigma ^{2}\mathbf {I} )}$. If the parameters ${\displaystyle \beta }$ and ${\displaystyle \sigma ^{2}}$ are stacked in the vector ${\displaystyle \mathbf {\theta } ^{\mathsf {T}}={\begin{bmatrix}\beta &\sigma ^{2}\end{bmatrix}}}$, the resulting log-likelihood function is

${\displaystyle \ell (\mathbf {\theta } )=-{\frac {n}{2}}\log \sigma ^{2}-{\frac {1}{2\sigma ^{2}}}\left(\mathbf {y} -\mathbf {X} \mathbf {\beta } \right)^{\mathsf {T}}\left(\mathbf {y} -\mathbf {X} \mathbf {\beta } \right)}$

The information matrix can then be expressed as

${\displaystyle \mathbf {I} (\mathbf {\theta } )=\operatorname {E} \left[\left({\frac {\partial \ell (\mathbf {\theta } )}{\partial \mathbf {\theta } }}\right)\left({\frac {\partial \ell (\mathbf {\theta } )}{\partial \mathbf {\theta } }}\right)^{\mathsf {T}}\right]}$

that is the expected value of the outer product of the gradient or score. Second, it can be written as the negative of the Hessian matrix of the log-likelihood function

${\displaystyle \mathbf {I} (\mathbf {\theta } )=-\operatorname {E} \left[{\frac {\partial ^{2}\ell (\mathbf {\theta } )}{\partial \mathbf {\theta } \,\partial \mathbf {\theta } ^{\mathsf {T}}}}\right]}$

If the model is correctly specified, both expressions should be equal. Combining the equivalent forms yields

${\displaystyle \mathbf {\Delta } (\mathbf {\theta } )=\sum _{i=1}^{n}\left[{\frac {\partial ^{2}\ell (\mathbf {\theta } )}{\partial \mathbf {\theta } \,\partial \mathbf {\theta } ^{\mathsf {T}}}}+{\frac {\partial \ell (\mathbf {\theta } )}{\partial \mathbf {\theta } }}{\frac {\partial \ell (\mathbf {\theta } )}{\partial \mathbf {\theta } }}\right]}$

where ${\displaystyle \mathbf {\Delta } (\mathbf {\theta } )}$ is an ${\displaystyle (r\times r)}$ random matrix, where ${\displaystyle r}$ is the number of parameters. White showed that the elements of ${\displaystyle n^{-1/2}\mathbf {\Delta } (\mathbf {\hat {\theta }} )}$, where ${\displaystyle \mathbf {\hat {\theta }} }$ is the MLE, are asymptotically normally distributed with zero means when the model is correctly specified. ^[2] In small samples, however, the test generally performs poorly. ^[3]

References

1. White, Halbert (1982). "Maximum Likelihood Estimation of Misspecified Models". Econometrica . 50 (1): 1–25. doi:10.2307/1912526. JSTOR   1912526.
2. Godfrey, L. G. (1988). Misspecification Tests in Econometrics. Cambridge University Press. pp. 35–37. ISBN   0-521-26616-5.
3. Orme, Chris (1990). "The Small-Sample Performance of the Information-Matrix Test". Journal of Econometrics . 46 (3): 309–331. doi:10.1016/0304-4076(90)90012-I.

Further reading

• Krämer, W.; Sonnberger, H. (1986). The Linear Regression Model Under Test. Heidelberg: Physica-Verlag. pp. 105–110. ISBN   3-7908-0356-1.
• White, Halbert (1994). "Information Matrix Testing". Estimation, Inference and Specification Analysis. New York: Cambridge University Press. pp. 300–344. ISBN   0-521-25280-6.