Visualization of learning in multilayer perceptron networks using principal component analysis

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 33, Issue 1, 2003, Pages 28-34

  Gallagher, Marcus ^a   Downs, Thomas ^a  

^a UNIVERSITY OF QUEENSLAND   (Australia)

Author keywords

Artificial neural network (ANN); Error surface; Multilayer perceptron; Principal component analysis (PCA); Visualization

Indexed keywords

ALGORITHMS; BACKPROPAGATION; FEEDFORWARD NEURAL NETWORKS; LEARNING SYSTEMS; PRINCIPAL COMPONENT ANALYSIS;

ERROR SURFACE;

MULTILAYER NEURAL NETWORKS;

EID: 0037275359     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCB.2003.808183     Document Type: Article

References (32)

1. R. Andrews, J. Diederich, and A. B. Tickle, "A survey and critique of techniques for extracting roles from trained artificial neural networks," Knowl.-Based Syst., vol. 8, no. 6, pp. 373-389, 1995.
2. G. S. Androulakis, G. D. Magoulas, and M. N. Vrahatis, "Geometry of learning: Visualizing the performance of neural network supervised training methods," Nonlinear Analysis, Theory, Methods, and Applications, vol. 30, no. 7, pp. 4359-4544, 1997.
3. C. M. Bishop, Neural Networks for Pattern Recognition. London, U.K.: Oxford Univ. Press, 1995.
4. C. Blake, E. Keogh, and C. J. Merz. (1998) UCI repository of machine learning databases. [Online]. Available: http://www.ics.uci.edu/̃mlearn/MLRepository.html
5. M. W. Craven and J. W. Shavlik, "Visualizing learning and computation in artificial neural networks," Computer Sciences Department, Univ. Wisconsin, Madison, Tech. Rep. 91-5, 1991.
6. B. S. Everitt, Graphical Techniques for Multivariate Data. London, U.K.: Heinemann, 1978.
7. M. Gallagher, "Multilayer perceptron error surfaces: visualization, structure and modeling," Ph.D. dissertation, Dept. Computer Science and Electrical Engineering, University of Queensland, Qld., U.K., 2000.
8. J. Hertz, A. Krogh, and R. G. Palmer, Introduction to the Theory of Neural Computation. Reading, MA: Addison-Wesley, 1991.
9. G. E. Hinton, J. L. McClelland, and D. E. Rumelhart, "Distributed representations," in Parallel Distributed Processing. Cambridge, MA: MIT Press, 1986, vol. 1, ch. 3, pp. 77-109.
10. D. R. Hush, B. Home, and J. M. Salas, "Error surfaces for multilayer perceptrons," IEEE Trans. Syst., Man, Cybern., vol. 22, no. 5, pp. 1152-1161, 1992.
11. E. J. Jackson, A User's Guide to Principal Components. New York: Wiley, 1991.
12. I. T. Jolliffe, Principal Component Analysis. New York: Springer-Verlag, 1986.
13. M. M. Kantardzic, A. A. Aly, and A. S. Elmaghraby, "Visualization of neural-network gaps based on error analysis," IEEE Trans. Neural Networks, vol. 10, no. 2, pp. 419-426, 1999.
14. S. Khuri and J. Williams, "Neuralis: An artificial neural network package," SIGCSE-Bull., vol. 28, pp. 25-27, 1996.
15. W. J. Krzanowski, Principles of Multivariate Analysis: A User's Perspective. Oxford, U.K.: Clarendon, 1988.
16. K. J. Lang and M. J. Witbrock, "Learning to tell two spirals apart," in Proc. Connectionist Models Summer School, T. Sejnowski, D. Touretzky, and G. Hinton, Eds., San Mateo, CA, 1988, pp. 52-59.
17. S. R. Lawrence, "Neural networks for real world tasks: limitations and solutions," Ph.D. dissertation, Univ. of Queensland, Qld., U.K., 1997.
18. Y. Liao and J. Moody, "A neural network visualization and sensitivity analysis toolkit," in Proc. ICONIP, 1996, pp. 1069-1074.
19. R. Lister, "Visualizing weight dynamics in the N-2-N encoder," in Proc. IEEE Int. Conf. Neural Networks, vol. 2, Piscataway, NJ, 1993, pp. 684-689.
20. B. F. J. Manly, Multivariate Statistical Methods: A Primer. London, U.K.: Chapman & Hall, 1994.
21. P. W. Munro, "Visualizations of 2-D hidden unit space," in Int. Joint Conf. Neural Networks, vol. III, New York, 1992, pp. 468-473.
22. N. J. Nilsson, Learning Machines. New York: McGraw-Hill, 1965.
23. S. Phillips, "The effect of representation on error surface," in Proc. 4th ACNN, P. Leong and M. Jabri, Eds., 1993, pp. 86-89.
24. A. J. Pinz and H. Bischof, "Constructing a neural network for the interpretation of the species of trees in aerial photographs," in Proc. 10th Int. Conf. Pattern Recognition, vol. 1, Los Alamos, CA, 1990, pp. 755-757.
25. T. Plate, J. Bert, J. Grace, and P. Band, "Visualizing the function computed by a feedforward neural network," Neural Comput., vol. 12, no. 6, pp. 1337-1354, 2000.
26. B. Ripley, Pattern Recognition and Neural Networks. Cambridge, U.K.: Cambridge Univ. Press, 1996.
27. R. Rojas, "Visualizing the learning process for neural networks," in Proc. ESANN, M. Verleysen, Ed., Brussels, Belgium, 1994, pp. 211-216.
28. 2 VIS: An interactive visualization tool for neural networks," in Proc. Visual Data Exploration and Analysis VIII, vol. 4302, Proc. SPIE, San Jose, CA, 2001.
29. E. van der Poel and I. Cloete, "Animating neural network training," South African Computer J., no. 7, pp. 44-52, 1992.
30. J. Wejchert and G. Tesauro, "Neural network visualization," in Proc. Advances in Neural Information Processing Systems, vol. 2, D. Touretzky, Ed., San Mateo, CA, 1990, pp. 465-472.
31. _, "Visualizing processes in neural networks," IBM J. Rex. Develop., vol. 35, no. 1/2, pp. 244-253, 1991.
32. P. Wilke, "Visualization of neural networks using NEUROGRAPH," University Education Uses of Visualization in Scientific Computing, no. A-48, pp. 105-117, 1994.