INFORMATION-BASED ALGORITHMIC DESIGN OF A NEURAL NETWORK CLASSIFIER
DOI:
https://doi.org/10.47839/ijc.5.3.412Keywords:
Information-based complexity, artificial neural network, adaptive, non-adaptive, canonical form, perceptron, clustersAbstract
An information-based design principle is presented that provides a framework for the design of both parallel and sequential algorithms. In this presentation, the notion of information (data) organization and canonical separation are examined and used in the design of an iterative line method for pattern grouping. In addition this technique is compared to the Winner Take All (WTA) method and shown to have many advantages.References
J. F. Traub, G. W. Wasilkowski, and H. Wozniakowski. Information-Based Complexity, Academic Press Series In Computer Science And Scientific Computing Archive, 1988.
B. N. Parlett. Some Basic Information on Information-Based Complexity Theory, Bulletin of the American Mathematical Society, 1992, Vol. 26, No. 1, pp. 3-28.
J. F. Traub and H. Wozniakowski. Perspectives on Information-Based Complexity, Bulletin of the American Mathematical Society, 1992, Vol. 26, No. 1, Pages 29-52.
M. H. Kalos and P. A. Whitlock. Monte Carlo Methods, Volume I: Basics, Wiley-Interscience Publications, John Wiley and Sons 1986, New York.
Doucet, N. de Freitas, and N. Gordon. Sequential Monte Carlo Methods in Practice, Springer 2001.
M. Gunzburger, R. E. Hiromoto, and M. Mundt. Analysis of a Monte Carlo Boundary Propagation Method, Journal of Computers and Math. with Applic. 1996, Vol. 31, No. 6, pp. 61-70.
Hecht-Nielsen, R. Counter-Propagation networks, IEEE First International Conference on Neural Networks, Volume II, 1987.
V. Maniezzo. Genetic evolution of the topology and weight distribution of neural networks, IEEE Transactions on Neural Networks, 1994, Vol. 5, No.1, pp. 39-53.
S. Mizuta , T. Sato, D. Lao, M. Ikeda, T. Shimizu. Structure design of neural networks using genetic algorithms, Complex Systems, 2001, 13, pp. 161-175.
K. Balakrishnan , V. Honavar. Evolutionary design of neural architectures – preliminary taxonomy and guide to literature, Artificial Intelligence Group, Iowa State University, Ames, Tech. Rep. CS TR#98-01, 1995.
Manic, M. Wilamowski, D. Robust Neural Network Training Using Partial Gradient Probing, IEEE Int. Conf. on Industrial Informatics, INDIN 2003, August 21-24, Banff, Alberta, Canada.
J. M. Zurada. Introduction to Artificial Neural Systems, West Publishing Company 1992.
Wilamowska, K., Manic, M. Unsupervised pattern clustering for data mining, IECON'01 - 27. Annual Conference of the IEEE Industrial Electronics Society, Denver, Colorado, Nov 29 to Dec 2, 2001, pp.1862-1867.
F. Rosenblatt. The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain, Psychological Review, 1958 v65, No. 6, pp. 386-408.
Fahlman and Lebiere. The Cascade-Correlation Learning Architecture, in Advances in Neural Information Processing Systems 2, D.Touretzky, ed., San Mateo, CA, Morgan Kaufmann, 1990, pp.524-532.
Marquardt, D. An Algorithm for Least-Squares Estimation of Nonlinear Parameters, SIAM J. Appl. Math. 1963, 11, 431-441.
Rumelhart, D.E., McClelland, J.L. (1986) Parallel Distributed Processing: Explorations in the Microstructure of Cognition, MIT Press Cambridge, MA. 1986, Vol. 1.
Rumelhart, D.E., Hinton, G.E., and Williams, R.J. Learning Internal Representation by Error Propagation, Parallel Distributed Processing, MIT Press, Cambridge, MA. 1986 Vol.1, pp.318-362.
Sejnowski T.J., Rosenberg, C.R. Parallel Networks that Learn to Pronounce English Text, Complex Systems, 1987, Vol. 1, 145-168.
Kohonen, T. Self-organized formation of topologically correct feature maps, in Biological Cybernetics, 1982, 43:59-69.
Kohonen, T. Self-Organization and Associative Memory, Springer-Verlag, 1988 2nd Ed. New York.
Downloads
Published
How to Cite
Issue
Section
License
International Journal of Computing is an open access journal. Authors who publish with this journal agree to the following terms:• Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
• Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
• Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
