INFORMATION-BASED ALGORITHMIC DESIGN OF A NEURAL NETWORK CLASSIFIER

Authors

  • Robert E. Hiromoto
  • Milos Manic

DOI:

https://doi.org/10.47839/ijc.5.3.412

Keywords:

Information-based complexity, artificial neural network, adaptive, non-adaptive, canonical form, perceptron, clusters

Abstract

An information-based design principle is presented that provides a framework for the design of both parallel and sequential algorithms. In this presentation, the notion of information (data) organization and canonical separation are examined and used in the design of an iterative line method for pattern grouping. In addition this technique is compared to the Winner Take All (WTA) method and shown to have many advantages.

References

J. F. Traub, G. W. Wasilkowski, and H. Wozniakowski. Information-Based Complexity, Academic Press Series In Computer Science And Scientific Computing Archive, 1988.

B. N. Parlett. Some Basic Information on Information-Based Complexity Theory, Bulletin of the American Mathematical Society, 1992, Vol. 26, No. 1, pp. 3-28.

J. F. Traub and H. Wozniakowski. Perspectives on Information-Based Complexity, Bulletin of the American Mathematical Society, 1992, Vol. 26, No. 1, Pages 29-52.

M. H. Kalos and P. A. Whitlock. Monte Carlo Methods, Volume I: Basics, Wiley-Interscience Publications, John Wiley and Sons 1986, New York.

Doucet, N. de Freitas, and N. Gordon. Sequential Monte Carlo Methods in Practice, Springer 2001.

M. Gunzburger, R. E. Hiromoto, and M. Mundt. Analysis of a Monte Carlo Boundary Propagation Method, Journal of Computers and Math. with Applic. 1996, Vol. 31, No. 6, pp. 61-70.

Hecht-Nielsen, R. Counter-Propagation networks, IEEE First International Conference on Neural Networks, Volume II, 1987.

V. Maniezzo. Genetic evolution of the topology and weight distribution of neural networks, IEEE Transactions on Neural Networks, 1994, Vol. 5, No.1, pp. 39-53.

S. Mizuta , T. Sato, D. Lao, M. Ikeda, T. Shimizu. Structure design of neural networks using genetic algorithms, Complex Systems, 2001, 13, pp. 161-175.

K. Balakrishnan , V. Honavar. Evolutionary design of neural architectures – preliminary taxonomy and guide to literature, Artificial Intelligence Group, Iowa State University, Ames, Tech. Rep. CS TR#98-01, 1995.

Manic, M. Wilamowski, D. Robust Neural Network Training Using Partial Gradient Probing, IEEE Int. Conf. on Industrial Informatics, INDIN 2003, August 21-24, Banff, Alberta, Canada.

J. M. Zurada. Introduction to Artificial Neural Systems, West Publishing Company 1992.

Wilamowska, K., Manic, M. Unsupervised pattern clustering for data mining, IECON'01 - 27. Annual Conference of the IEEE Industrial Electronics Society, Denver, Colorado, Nov 29 to Dec 2, 2001, pp.1862-1867.

F. Rosenblatt. The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain, Psychological Review, 1958 v65, No. 6, pp. 386-408.

Fahlman and Lebiere. The Cascade-Correlation Learning Architecture, in Advances in Neural Information Processing Systems 2, D.Touretzky, ed., San Mateo, CA, Morgan Kaufmann, 1990, pp.524-532.

Marquardt, D. An Algorithm for Least-Squares Estimation of Nonlinear Parameters, SIAM J. Appl. Math. 1963, 11, 431-441.

Rumelhart, D.E., McClelland, J.L. (1986) Parallel Distributed Processing: Explorations in the Microstructure of Cognition, MIT Press Cambridge, MA. 1986, Vol. 1.

Rumelhart, D.E., Hinton, G.E., and Williams, R.J. Learning Internal Representation by Error Propagation, Parallel Distributed Processing, MIT Press, Cambridge, MA. 1986 Vol.1, pp.318-362.

Sejnowski T.J., Rosenberg, C.R. Parallel Networks that Learn to Pronounce English Text, Complex Systems, 1987, Vol. 1, 145-168.

Kohonen, T. Self-organized formation of topologically correct feature maps, in Biological Cybernetics, 1982, 43:59-69.

Kohonen, T. Self-Organization and Associative Memory, Springer-Verlag, 1988 2nd Ed. New York.

Downloads

Published

2014-08-01

How to Cite

Hiromoto, R. E., & Manic, M. (2014). INFORMATION-BASED ALGORITHMIC DESIGN OF A NEURAL NETWORK CLASSIFIER. International Journal of Computing, 5(3), 87-98. https://doi.org/10.47839/ijc.5.3.412

Issue

Section

Articles