Toward Bayes-Optimal Linear Dimension Reduction
April 1994 (vol. 16 no. 4) pp. 420-424
Dimension reduction is the process of transforming multidimensional vectors into a low-dimensional space. In pattern recognition, it is often desired that this task be performed without significant loss of classification information. The Bayes error is an ideal criterion for this purpose; however, it is known to be notoriously difficult for mathematical treatment. Consequently, suboptimal criteria have been used in practice. We propose an alternative criterion, based on the estimate of the Bayes error, that is hopefully closer to the optimal criterion than the criteria currently in use. An algorithm for linear dimension reduction, based on this criterion, is conceived and implemented. Experiments demonstrate its superior performance in comparison with conventional algorithms.
[1] 420J. A. Nelder and R. Mead, "A simplex method for function minimization,"Comput. J., vol. 8, pp. 308-313, 1965.
[2] L. F. Guseman, Jr., B. C. Peters, Jr., and H. F. Walker, "On minimizing the probability of misclassification for linear feature selection,"Annals Statist., vol. 3, pp. 661-668, 1975.
[3] S. A. Dudani, K. J. Breeding, and R. B. McGhee, "Aircraft identification by moment invariants,"IEEE Trans. Comput., vol. C-26, pp. 39-45, 1977.
[4] A. K. Jain and W. G. Waller, "On the optimal number of features in the classification of multivariate Gaussian data,"Pattern Recogn., vol. 10, pp. 365-374, 1978.
[5] S. J. Raudys, "Determination of optimal dimensionality in statistical pattern classification,"Pattern Recogn., vol. 11, pp. 263-270, 1979.
[6] G. Biswas, A. K. Jain, and R. C. Dubes, "Evaluation of projection algorithms,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-3, pp. 701-708, 1981.
[7] P. A. Devijver and J. Kittler,Pattern Recognition: A Statistical Approach. Englewood Cliffs, NJ: Prentice-Hall International, 1982.
[8] J. Kittler, "Feature selection and extraction," inHandbook of Pattern Recognition and Image Processing, T. Y. Young and K.-S. Fu, Eds. Orlando, FL: Academic, 1986.
[9] K. Fukunaga and D. Hummels, "Bayes error estimation using Parzen andk-NN procedure,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI- 9, pp. 634-643, 1987.
[10] R. P. Gorman and T. J. Sejnowski, "Analysis of hidden units in a layered network trained to classify sonar targets,"Neural Networks, vol. 1, pp. 75-89, 1988.
[11] K. Fukunaga,Introduction to Statistical Pattern Recognition. New York: Academic, 1972.
[12] P. Pejnovic´, "Application of moment invariants in visual pattern recognition" (in Serbo-Croatian), M.S. thesis, Faculty of Electrical Engineering, Univ. of Belgrade, Belgrade, Yugoslavia, 1991.
[13] Lj. Buturovic´and Z. Stojiljkovic´, "PARIS--pattern analysis and recognition interactive system,"Pattern Recogn., vol. 24, p. 11, 1991.
[14] Lj. J. Buturovic´, "On the minimal dimension of sufficient statistics,"IEEE Trans. Inform. Theory, vol. 38, pp. 182-186, 1992.
Index Terms:
pattern recognition; statistical analysis; probability; Bayes methods; optimisation; estimation theory; Bayes optimal linear dimension reduction; multidimensional vectors; low dimensional space; statistical pattern recognition; Bayes error estimation; probability density; k-nearest neighbour; optimisation
Citation:
L.J. Buturovic, "Toward Bayes-Optimal Linear Dimension Reduction," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 16, no. 4, pp. 420-424, Apr., 1994
PURCHASE ARTICLE: $19
IEEE Xplore Subscribers
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb