Selection of proper neural network sizes and architectures-A comparative study

IEEE Transactions on Industrial Informatics

Volumn 8, Issue 2, 2012, Pages 228-240

  Hunter, David ^a   Yu, Hao ^a   Pukish III, Michael S ^a   Kolbusz, Janusz ^b   Wilamowski, Bogdan M ^a  

^a AUBURN UNIVERSITY   (United States)

^b UNIVERSITY OF INFORMATION TECHNOLOGY AND MANAGEMENT   (Poland)

Author keywords

Architectures; learning algorithms; neural networks; topologies

Indexed keywords

BENCH-MARK PROBLEMS; COMPARATIVE STUDIES; ERROR BACK PROPAGATION ALGORITHM; LEVENBERG-MARQUARDT ALGORITHM; MULTI LAYER PERCEPTRON; NETWORK TOPOLOGY; NEURAL NETWORK SIZE; TRAINING PROCESS;

ARCHITECTURE; ELECTRIC NETWORK TOPOLOGY; LEARNING ALGORITHMS; NETWORK ARCHITECTURE; TOPOLOGY;

NEURAL NETWORKS;

EID: 84859886103     PISSN: 15513203     EISSN: None     Source Type: Journal    
DOI: 10.1109/TII.2012.2187914     Document Type: Article

References (71)

1. B. K. Bose, "Neural network applications in power electronics and motor drives-An introduction and perspective," IEEE Trans. Ind. Electron., vol. 54, no. 1, pp. 14-33, Feb. 2007. (Pubitemid 46444002)
2. T. Orlowska-Kowalska, M. Dybkowski, and K. Szabat, "Adaptive sliding-mode neuro-fuzzy control of the two-mass induction motor drive without mechanical sensors," IEEE Trans. Ind. Electron., vol. 57, no. 2, pp. 553-564, Feb. 2010.
3. M. Pucci and M. Cirrincione, "Neural MPPT control of wind generators with induction machines without speed sensors," IEEE Trans. Ind. Electron., vol. 58, no. 1, pp. 37-47, Jan. 2011.
4. V. N. Ghate and S. V. Dudul, "Cascade neural-network-based fault classifier for three-phase induction motor," IEEE Trans. Ind. Electron., vol. 58, no. 5, pp. 1555-1563, May 2011.
5. C. Xia, C. Guo, and T. Shi, "A neural-network-identifier and fuzzycontroller- based algorithm for dynamic decoupling control of permanent- magnet spherical motor," IEEE Trans. Ind. Electron., vol. 57, no. 8, pp. 2868-2878, Aug. 2010.
6. F. F. M. El-Sousy, "Hybri-based wavelet-neural-network tracking control for permanent-magnet synchronous motor servo drives," IEEE Trans. Ind. Electron., vol. 57, no. 9, pp. 3157-3166, Sep. 2010.
7. F. F. M. El-Sousy, "Hybrid based wavelet-neural-network tracking control for permanent-magnet synchronous motor servo drives," IEEE Trans. Ind. Electron., vol. 57, no. 9, pp. 3157-3166, Sep. 2010.
8. Q. N. Le and J. W. Jeon, "Neural-network-based low-speed-damping controller for stepper motor with an FPGA," IEEE Trans. Ind. Electron., vol. 57, no. 9, pp. 3167-3180, Sep. 2010.
9. C.-H. Lu, "Wavelet fuzzy neural networks for identification and predictive control of dynamic systems," IEEE Trans. Ind. Electron., vol. 58, no. 7, pp. 3046-3058, Jul. 2011.
10. R. H. Abiyev and O. Kaynak, "Type 2 fuzzy neural structure for identification and control of time-varying plants," IEEE Trans. Ind. Electron., vol. 57, no. 12, pp. 4147-4159, Dec. 2010.
11. M. A. S. K. Khan and M. A. Rahman, "Implementation of a waveletbased MRPID controller for benchmark thermal system," IEEE Trans. Ind. Electron., vol. 57, no. 12, pp. 4160-4169, Dec. 2010.
12. A. Bhattacharya and C. Chakraborty, "A shunt active power filter with enhanced performance using ANN-based predictive and adaptive controllers," IEEE Trans. Ind. Electron., vol. 58, no. 2, pp. 421-428, Feb. 2011.
13. J. Lin and R.-J. Lian, "Intelligent control of active suspension systems," IEEE Trans. Ind. Electron., vol. 58, no. 2, pp. 618-628, Feb. 2011.
14. Z. Lin, J. Wang, and D. Howe, "A learning feed-forward current controller for linear reciprocating vapor compressors," IEEE Trans. Ind. Electron., vol. 58, no. 8, pp. 3383-3390, Aug. 2011.
15. M. O. Efe, "Neural network assisted computationally simple PI D control of a quadrotor UAV," IEEE Trans. Ind. Informat., vol. 7, no. 2, pp. 354-361, May 2011.
16. T. Orlowska-Kowalska and M. Kaminski, "FPGA implementation of the multilayer neural network for the speed estimation of the two-mass drive system," IEEE Trans. Ind. Informat., vol. 7, no. 3, pp. 436-445, Aug. 2011.
17. C.-F. Juang, Y.-C. Chang, and C.-M. Hsiao, "Evolving gaits of a hexapod robot by recurrent neural networks with symbiotic species-based particle swarm optimization," IEEE Trans. Ind. Electron., vol. 58, no. 7, pp. 3110-3119, Jul. 2011.
18. C.-C. Tsai, H.-C. Huang, and S.-C. Lin, "Adaptive neural network control of a self-balancing two-wheeled scooter," IEEE Trans. Ind. Electron., vol. 57, no. 4, pp. 1420-1428, Apr. 2010.
19. G. W. Chang, C.-I. Chen, and Y.-F. Teng, "Radial-basis-function- based neural network for harmonic detection," IEEE Trans. Ind. Electron., vol. 57, no. 6, pp. 2171-2179, Jun. 2010.
20. R.-J. Wai and C.-Y. Lin, "Active low-frequency ripple control for clean-energy power-conditioning mechanism," IEEE Trans. Ind. Electron., vol. 57, no. 11, pp. 3780-3792, Nov. 2010.
21. Yahyaoui, N. Fnaiech, and F. Fnaiech, "A suitable initialization procedure for speeding a neural network job-shop scheduling," IEEE Trans. Ind. Electron., vol. 58, no. 3, pp. 1052-1060, Mar. 2011.
22. V. Machado, A. Neto, and J. D. de Melo, "A neural network multiagent architecture applied to industrial networks for dynamic allocation of control strategies using standard function blocks," IEEE Trans. Ind. Electron., vol. 57, no. 5, pp. 1823-1834, May 2010.
23. M. Charkhgard and M. Farrokhi, "State-of-charge estimation for lithium-ion batteries using neural networks and EKF," IEEE Trans. Ind. Electron., vol. 57, no. 12, Dec. 2010.
24. M. Wilamowski and O. Kaynak, "Oil well diagnosis by sensing terminal characteristics of the induction motor," IEEE Trans. Ind. Electron., vol. 47, no. 5, pp. 1100-1107, Oct. 2000.
25. N. J. Cotton and B. M. Wilamowski, "Compensation of nonlinearities using neural networks implemented on inexpensive microcontrollers," IEEE Trans. Ind. Electron., vol. 58, no. 3, pp. 733-740, Mar. 2011.
26. A. Gomperts, A. Ukil, and F. Zurfluh, "Development and implementation of parameterized FPGA-based general purpose neural networks for online applications," IEEE Trans. Ind. Informat., vol. 7, no. 1, pp. 78-89, Feb. 2011.
27. E. Monmasson, L. Idkhajine, M. N. Cirstea, I. Bahri, A. Tisan, and M. W. Naouar, "FPGAs in industrial control applications," IEEE Trans. Ind. Informat., vol. 7, no. 2, pp. 224-243, May 2011.
28. A. Dinu, M. N. Cirstea, and S. E. Cirstea, "Direct neural-network hardware- implementation algorithm," IEEE Trans. Ind. Electron., vol. 57, no. 5, pp. 1845-1848, May 2010.
29. A. Malinowski and H. Yu, "Comparison of various embedded system technologies for industrial applications," IEEE Trans. Ind. Informat., vol. 7, no. 2, pp. 244-254, May 2011.
30. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning representations by back-propagating errors," Nature, vol. 323, pp. 533-536, 1986. (Pubitemid 16025374)
31. P. J. Werbos, "Back-propagation: Past and future," in Proc. Int. Conf. Neural Netw., San Diego, CA, 1988, pp. 343-354, 1.
32. S. Ferrari and M. Jensenius, "A constrained optimization approach to preserving prior knowledge during incremental training," IEEE Trans. Neural Netw., vol. 19, no. 6, pp. 996-1009, Jun. 2008.
33. Q. Song, J. C. Spall, Y. C. Soh, and J. Ni, "Robust neural network tracking controller using simultaneous perturbation stochastic approximation," IEEE Trans. Neural Netw., vol. 19, no. 5, pp. 817-835, May 2008. (Pubitemid 351728818)
34. A. Slowik, "Application of an adaptive differential evolution algorithm with multiple trial vectors to artificial neural network training," IEEE Trans. Ind. Electron., vol. 58, no. 8, pp. 3160-3167, Aug. 2011.
35. V. V. Phansalkar and P. S. Sastry, "Analysis of the back-propagation algorithm with momentum," IEEE Trans. Neural Netw., vol. 5, no. 3, pp. 505-506, Mar. 1994.
36. A. Salvetti and B. M. Wilamowski, "Introducing stochastic processes within the backpropagation algorithm for improved convergence," in Proc. Artif. Neural Netw. Eng.-ANNIE'94.
37. B. M. Wilamowski and L. Torvik, "Modification of gradient computation in the back-propagation algorithm," in Artif. Neural Netw. Eng.-ANNIE'93.
38. M. Riedmiller and H. Braun, "A direct adaptive method for faster backpropagation learning: The RPROP algorithm," in Proc. Int. Conf. Neural Netw., San Francisco, CA, 1993, pp. 586-591.
39. S. E. Fahlman, An Empirical Study of Learning Speed in Backpropagation Networks 1988.
40. K. Levenberg, "A method for the solution of certain problems in least squares," Quart. Appl. Mach., vol. 5, pp. 164-168, 1944.
41. M. T. Hagan and M. B. Menhaj, "Training feedforward networks with the Marquardt algorithm," IEEE Trans. Neural Netw., vol. 5, no. 6, pp. 989-993, Nov. 1994.
42. B. M. Wilamowski, "Neural network architectures and learning algorithms," IEEE Ind. Electron. Mag., vol. 3, no. 4, pp. 56-63, Dec. 2009.
43. B. M. Wilamowski, "Challenges in applications of computational intelligence in industrial electronics," in Proc. Int. Symp. Ind. Electron.- ISIE'10, Bari, Italy, Jul. 4-7, 2010, pp. 15-22.
44. N. J. Cotton, B. M. Wilamowski, and G. Dundar, "A neural network implementation on an inexpensive eight bit microcontroller," in Proc. 12th Int. Conf. Intell. Eng. Syst.-INES'08, Miami, Florida, USA, Feb. 25-29, 2008, pp. 109-114.
45. B. M. Wilamowski, N. J. Cotton, O. Kaynak, and G. Dundar, "Computing gradient vector and jacobian matrix in arbitrarily connected neural networks," IEEE Trans. Ind. Electron., vol. 55, no. 10, pp. 3784-3790, Oct. 2008.
46. B. M. Wilamowski and H. Yu, "Improved computation for Levenberg Marquardt training," IEEE Trans. Neural Netw., vol. 21, no. 6, pp. 930-937, Jun. 2010.
47. B. M.Wilamowski and H. Yu, "Neural network learning without backpropgation," IEEE Trans. Neural Netw., vol. 21, no. 11, pp. 1793-1803, Nov. 2010.
48. J. R. Alvarez-Sanchez, "Injecting knowledge into the solution of the two-spiral problem," Neural Comput. Appl., vol. 8, pp. 265-272, Aug. 1999.
49. S. Wan and L. E. Banta, "Parameter incremental learning algorithm for neural networks," IEEE Trans. Neural Netw., vol. 17, no. 6, pp. 1424-1438, Jun. 200.
50. S. E. Fahlman and C. Lebiere, , D. S. Touretzky, Ed., "The cascadecorrection learning architecture," in Advances in Neural Information Processing Systems 2. San Mateo, CA: Morgan Kaufmann.
51. H. Yu, T. T. Xie, S. Paszczynski, and B. M.Wilamowski, "Advantages of radial basis function networks for dynamic system design," IEEE Trans. Ind. Electron., vol. 58, no. 12, pp. 5438-5450, Dec. 2011.
52. B. M. Wilamowski and L. Torvik, "Modification of gradient computation in the back-propagation algorithm," in Proc. Artif. Neural Netw. Eng.-ANNIE'93, St. Louis, MO, Nov. 14-17, 1993, pp. 175-180.
53. B. M. Wilamowski, D. Hunter, and A. Malinowski, "Solving parity-n problems with feedforward neural network," in Proc. Int. Joint Conf. Neural Netw.-IJCNN'03, Portland, OR, Jul. 20-23, 2003, pp. 2546-2551.
54. S. Trenn, "Multilayer perceptrions: Approximation order and necessary number of hidden units," IEEE Trans. Neural Netw., vol. 19, no. 5, pp. 836-844, May 2008. (Pubitemid 351728819)
55. D. Hunter and B. M. Wilamowski, "Parallel multi-layer neural network architecture with improved efficiency," in Proc. 4th Conf. Human Syst. Interactions-HSI'11, Yokohama, Japan, May 19-21, 2011, pp. 299-304.
56. B. M. Wilamowski, H. Yu, and K. Chung, "Parity-N problems as a vehicle to compare efficiency of neural network architectures," in Industrial Electronics Handbook, 2nd ed. Boca Raton, FL: CRC Press, 2011, vol. 5, Intelligent Systems, ch. 10, pp. 10-1-10-8.
57. T. Sugeno and G. T. Kang, "Structure identification of fuzzy model," Fuzzy Sets and Systems, vol. 28, no. 1, pp. 15-33, 1988.
58. T. T. Xie, H. Yu, and B. M. Wilamowski, "Replacing fuzzy systems with neural networks," in Proc. IEEE Human Syst. Interaction Conf. -HSI'10, Rzeszow, Poland, May 13-15, 2010, pp. 189-193.
59. H. B. Demuth and M. Beale, Neural Network Toolbox: For Use With MATLAB. Natick, MA: Mathworks, 2000.
60. H. Yu and B. M.Wilamowski, "Efficient and reliable training of neural networks," in Proc. IEEE Human Syst. Interaction Conf.-HSI'09, Catania, Italy, May 21-23, 2009, pp. 109-115.
61. T. M. Cover, "Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition," IEEE Trans. Electron. Comput., vol. EC-14, pp. 326-334.
62. T. T. Xie, H. Yu, and B. M. Wilamowski, "Comparison of traditional neural networks and radial basis function networks," in Proc. 20th IEEE Int. Symp. Ind. Electron.-ISIE'11, Gdansk, Poland, Jun. 27-30, 2011, pp. 1194-1199.
63. S. Chen, C. F. N. Cowan, and P. M. Grant, "Orthogonal least squares learning algorithm for radial basis function networks," IEEE Trans. Neural Netw., vol. 2, no. 2, pp. 302-309, Mar. 1991.
64. G. B. Huang, P. Saratchandran, and N. Sundararajan, "A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation," IEEE Trans. Neural Netw., vol. 16, no. 1, pp. 57-67, Jan. 2005. (Pubitemid 40241910)
65. M. Pucci and M. Cirrincione, "Neural MPPT control of wind generators with induction machines without speed sensors," IEEE Trans. Ind. Electron., vol. 58, no. 1, pp. 37-47, Jan. 2011.
66. Q. N. Le and J. W. Jeon, "Neural-network-based low-speed-damping controller for stepper motor with an FPGA," IEEE Trans. Ind. Electron., vol. 57, no. 9, pp. 3167-3180, Sep. 2010.
67. K. Chen and A. Salman, "Learning speaker-specific characteristics with a deep neural architecture," IEEE Trans. Neural Netw., vol. 22, no. 11, pp. 1744-1756, 2011.
68. I. Arel, D. C. Rose, and T. P. Karnowski, "Deep machine learning-A new frontier in artificial intelligence research," IEEE Comput. Intell. Mag., vol. 5, no. 4, pp. 13-18, 2010.
69. Intelligent Engineering Systems Through Artificial Neural Networks. New York: ASME, 1994, vol. 4, pp. 205-209.
70. Intelligent Engineering Systems Through Artificial Neural Networks. New York: ASME, 1993, vol. 3, pp. 175-180, Also in.
71. Intelligent Engineering Systems Through Artificial Neural Networks. New York: ASME, 1993, vol. 3, pp. 175-180.