You are here

Home » Content

Misspecified Multivariate Regression Models Using the Genetic Algorithm and Information Complexity as the Fitness Function

Journal Name:

  • European Journal of Pure and Applied Mathematics

Publication Year:

  • 2012

Key Words:

Author NameUniversity of Author

AMS Codes:

Abstract (2. Language): 
Model misspecification is a major challenge faced by all statistical modeling techniques. Real world multivariate data in high dimensions frequently exhibit higher kurtosis and heavier tails, asymmetry, or both. In this paper, we extend Akaike’s AIC-type model selection criteria in two ways. We use a more encompassing notion of information complexity (ICOMP) of Bozdogan for multivariate regression to allow certain types of model misspecification to be detected using the newly proposed criterion so as to protect the researchers against model misspecification. We do this by employing the “sandwich”or “robust”covariance matrix Fˆ−1RˆFˆ−1, which is computed with the sample kurtosis and skewness. Thus, even if the data modeled do not meet the standard Gaussian assumptions, an appropriate model can still be found. Theoretical results are then applied to multivariate regression models in subset selection of the best predictors in the presence of model misspecification by using the novel genetic algorithm (GA), with our extended ICOMP as the fitness function. We demonstrate the power of the confluence of these techniques on both simulated and real-world datasets. Our simulations are very challenging, combining multicolinearity, unnecessary variables, and redundant variables with asymmetrical or leptokurtic behavior. We also demonstrate our model selection prowess on the well-known body fat data. Our findings suggest that when data are overly peaked or skewed - both characteristics often seen in real data, ICOMP based on the sandwich covariance matrix should be used to drive model selection.
Bookmark/Search this post with
FULL TEXT (PDF): 
PDF icon arastirmax-misspecified-multivariate-regression-models-using-genetic-algorithm-and-information-complexity-fitness-function.pdf
  • 2
211-249
  • English

REFERENCES

References: 

[1] H. Akaike. Information Theory and an Extension of the Maximum Likelihood Principle.
In B.N. Petrox and F. Csaki, editors, Second International Symposium on Information
Theory, pages 267–281, Budapest, 1973. Academiai Kiado.
[2] A. Azzalini. A Class of Distributions which Includes the Normal Ones. Scandinavian
Journal of Statistics, 12:171–178, 1985.
[3] G. Box and D. Cox. An Analysis of Transformations. Journal of the Royal Statistical
Society, Series B (Methodological), 26:211–246, 1964.
[4] H. Bozdogan. ICOMP: A New Model-Selection Criteria. In Classification and Related
Methods of Data Analysis.
[5] H. Bozdogan. Model Selection and Akaike’s Information Criteria (AIC): the General
Theory and its Analytical Extensions. Psychometrica, 52:317–332, 1987.
[6] H. Bozdogan. On the Information-Based Measure of Covariance Complexity and its
Application to the Evaluation of Multivariate Linear Models. Communication in Statistics,
Theory and Methods, 19:221–278, 1990.
[7] H. Bozdogan. Akaike’s Information Criterion and Recent Developments in Information
Complexity. Journal of Mathematical Psychology, 44:62–91, 2000.
[8] H. Bozdogan. A New Class of Information Complexity (ICOMP) Criteria with an Application
to Customer Profiling and Segmentation; an Invited Paper. Istanbul University
Journal of the School of Business Administration, 39(2):370–398, 2010.
[9] H. Bozdogan and D. Haughton. Informational Complexity Criteria for Regression Models.
Computational Statistics and Data Analysis, 28:51–76, 1998.
[10] K. Chaloner and I. Verdinelli. Bayesian Experimental Design: A Review. Statistical
Science, 10(3):273–304, August 1995.
[11] M. Chen. Estimation of Covariance Matrices Under a Quadratic Loss Function. Research
Report S-46, Department of Mathematics, SUNY at Albany, 1976.
[12] H. Cramér. Mathematical Methods of Statistics. Princeton University Press, Princeton,
New Jersey, 1946.
[13] A. Davison. Statistical Models. Cambridge University Press, Cambridge, 2003.
[14] B. Frieden. Physics from Fisher Information. Cambridge University Press, Cambridge,
UK, 1998.
[15] L. Godfrey. Misspecification Tests in Econometrics. Cambridge University Press, Cambridge,
1988.
[16] E. Gomez, M. Gomez-Villegas, and J. Marin. A multivariate generalization of the power
exponential family of distributions. Communications in Statistics - Theory and Methods,
27(3):589–600, 1998.
[17] C. Gouriéroux and A. Monfort. Statistics and Econometric Models, volume 1. Camridge
University Press, Cambridge, 1995.
[18] C. Gouriéroux and A. Monfort. Statistics and Econometric Models, volume 2. Camridge
University Press, Cambridge, 1995.
[19] D. Hendry. Dynamic Econometrics. Oxford University Press, Oxford, 1995.
[20] J. Hosking. Lagrange-Multiplier Tests of Time-Series Models. Journal of the Royal Statistical
Society, Series B (Methodological), 42:170–181, 1980.
[21] R. Kass, L. Tierney, and J. Kadane. Bayesian and Likelihood Methods in Statistics and
Econometrics, chapter The Validity of Posterior Expansions Based on Laplace’s Method,
pages 473–488. 1990.
[22] G. Kitagawa and S. Konishi. Bias and Variance Reduction Techniques for Bootstrap Information
Criteria. Annals of the Institute of Statistical Mathematics, 62:209–234, 2010.
[23] S. Konishi and G. Kitagawa. Information Criteria and Statistical Modeling. Springer, New
York, 2008.
[24] A. Kullback and R. Leibler. On Information and Sufficiency. Annals of Mathematical
Statistics, 22:79–86, 1951.
[25] D. Lindley. On a Measure of the Information Provided by an Experiment. The Annals of
Mathematical Statistics, 27(4):986–1005, December 1956.
[26] J. Magnus. Linear Structures. Charles Griffin & Company and Oxford University Press,
London, 1988.
[27] J. Magnus. The Asymptotic Variance of the Pseudo Maximum Likelihood Estimator.
Econometric Theory, 23:1022–1032, 2007.
[28] J. Magnus and H. Neudecker. Matrix Differential Calculus with Applications in Statistis
and Econometrics. Wiley, 1988.
[29] K. Mardia. Applications of some measures of multivariate skewness and kurtosis in
testing normality and robustness studies. Sankhya, B36:115–128, 1974.
[30] D. Poskitt. Precision, Complexity and Bayesian Model Determination. Journal of the
Royal Statistical Society, Series B (Methodological), 49(2):199–208, 1987.
[31] S. Press. Estimation of a Normal Covariance Matrix. Technical report, University of
British Columbia, 1975.
[32] C. Rao. Information and Accuracy Attainable in the Estimation of Statistical Parameters.
In Bulletin of the Calcutta Math Society, volume 37, page 81, 1945.
[33] C Rao. Minimum Variance and the Estimation of Several Parameters. In Proceedings of
the Cambridge Philosophical Society, volume 43, page 280, 1947.
[34] C. Rao. Sufficient Statistics and Minimum Variance Estimates. In Proceedings of the
Cambridge Philosophical Society, volume 45, page 213, 1948.
[35] J. Rissanen. Modeling by Shortest Data Description. Automatica, 14:465–471, 1978.
[36] T. Sawa. Information Criteria for Discriminating Among Alternative Regression Models.
Econometrica, 46:1273–1291, 1978.
[37] G. Schwarz. Estimating the Dimension of a Model. Annals of Statistics, 6:461–464, 1978.
[38] R. Shibata. Statistical Aspects of Model Selection. In J.C. Williams, editor, From Data to
Modeling, pages 216–240. Springer, Berlin, 1989.
[39] A. Shurygin. The Linear Combination of the Simplest Discriminator and Fisher’s One.
In Nauka, editor, Applied Statistics. Moscow, Russia, 1983.
[40] K. Takeuchi. Distribution of Information Statistics and Criterion of Model Fitting. Suri-
Kagaku (Mathematical Sciences), 153:12–18, 1976. In Japanese.
[41] C. Thomaz. Maximum Entropy Covariance Estimate for Statistical Pattern Recognition.
PhD thesis, University of London and for the Diploma of the Imperial College (D.I.C.),
2004.
[42] M. Van Emden. An Analysis of Complexity. In Mathematical Centre Tracts, volume 35.
Mathematisch Centrum, 1971.
[43] C. Wei. On Predictive Least Squares Principles. Annals of Statistics, 20:1–42, 1992.
[44] H. White. Maximum Likelihood Estimation of Misspecified Models. Econometrica, 50:1–
25, 1982.
[45] H. White. Estimation, Inference, and Specification Analysis. Cambridge University Press,
Cambridge, 1994.

Thank you for copying data from http://www.arastirmax.com